Log File Connector with delimited data
The Log File Connector sends a delimited message to a log file for each event bus hit. The message is formatted with a delimiter as specified by the configuration.
- If the text in the message contains the delimiter, then the character
is URL encoded in the string. For example, the comma character (
,
) would be encoded as%2C
. - The line length limit is 10,000 characters, including the delimiters. Note: This output connector should be implemented only on a dedicated file server with a Discover pipeline. This connector allows several input sources to write to the same log file at one time and frees up disk I/O from the main Discover system.
Below the FILE_Directory
, a new directory is created
for each day of data with a directory name in YYYYMMDD
format.
The log file names have the following format:
TL_EB_0_YYYYMMDD_mmmm.log
Where:
YYYY
- yearMM
- monthDD
- daymmmm
- Number of minutes past midnight for current day: 0000 - 1439
These files are structured in the following manner:
OutputType=FILE
File Delim: any character or TAB (note: not CSV compliant)
FILE_Delim=|
FILE_Directory=c:\Discover\logfiles
Row Time in minutes - log will be rolled every N minutes
FILE_RollTime=5
Log File schema
The Log connector uses the default schema.