Configuring Splunk - Implementing Splunk: Big Data Reporting and Development for Operational Intelligence - page 289

° DATETIME_CONFIG = /etc/apps/a/custom_datetime.xml : As
mentioned, Splunk uses a set of regular expressions to determine the
date. If TIME_FORMAT is not specified, or won't work for some strange
reason, you can specify a different set of regular expressions or
disable time extraction completely by setting this attribute to CURRENT
(the indexer clock time) or NONE (file modification time, or if there is
no file, clock time). I personally have never had to resort to a custom
datetime.xml file, though I have heard of it being done.
• The Data preview function available when adding data through the manager
interface builds a good, usable configuration. The generated configuration
does not use LINE_BREAKER , which is definitely safer but less efficient. Here
is a sample stanza using LINE_BREAKER for efficiency:
[mysourcetype]
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N %:z
MAX_TIMESTAMP_LOOKAHEAD = 32
TIME_PREFIX = ^\[
SHOULD_LINEMERGE = False
LINE_BREAKER = ([\r\n]+)(?=\[\d{4}-\d{1,2}-\d{1,2}\s+\
d{1,2}:\d{1,2}:\d{1,2})
TRUNCATE = 1024000
This configuration would apply to log messages that looked like this:
[2011-10-13 13:55:36.132 -07:00] ERROR Interesting message.
More information.
And another line.
[2011-10-13 13:55:36.138 -07:00] INFO All better.
[2011-10-13 13:55:37.010 -07:00] INFO More data
and another line.
Let's step through how these settings affect the first line of this sample configuration:
• LINE_BREAKER states that a new event starts when one or more newline
characters is followed by a bracket and series of numbers and dashes,
in the pattern [1111-11-11 11:11:11] .
• SHOULD_LINEMERGE=False tells Splunk to not bother trying to recombine
multiple lines.
• TIME_PREFIX moves the cursor to the character after the [ character.