Tables, Charts, and Fields - Implementing Splunk: Big Data Reporting and Development for Operational Intelligence - page 73

Defining an indexed field only makes sense if you know the format of
the logs before indexing, if you believe the filed will actually make the
query more efficient (see previous section), and if you will be searching
for the field value. If you will only be reporting on the values of this
field, an extracted field will be sufficient, except in the most extreme
performance cases.
Indexed field case 3 - application from source
A common requirement is to be able to search for events from a particular web
application. Often, the only easy way to determine the application that created the
logs is by inspecting the path to the logs, which Splunk stores in the indexed field
source . For example, given the following path, the application name is app_one :
/opt/instance19/apps/app_one/logs/important.log
You could search for this instance using source="*/app_one/*" , but this effectively
initiates a full table scan. You could define an extracted field and then search for
app="app_one" , but unfortunately, this approach will be no more efficient because
the word we're looking for is not contained in the field _raw . If we define this field as
an indexed field, app="app_one" will be an efficient search.
Once again, if you only need this field for reporting, the extracted field is just fine.
Indexed field case 4 - slow requests
Consider a web access log with a trailing request time in microseconds:
[31/Jan/2012:18:18:07 +0000] "GET / HTTP/1.1" 200 7918 ""
"Mozilla/5.0..." 11/11033255
Let's say we want to find all requests that took longer than 10 seconds. We can easily
extract the value into a field, perhaps request_ms . We could then run the search
request_ms>10000000 . This query will work, but it requires scanning every event
in the given time frame. Whether the field is extracted or indexed, we would face the
same problem as Splunk has to convert the field value to a number before it can test
the value.
What if we could define a field and instead search for slow_request=1 ? To do this,
we can take advantage of the fact that, when defining an indexed field, the value can
be a static value. This could be accomplished with a transform, like so:
REGEX = .*/(\d{7,})$
FORMAT = slow_request::1