Advanced Deployments - Implementing Splunk: Big Data Reporting and Development for Operational Intelligence - page 331

Splunk search
When there is only one Splunk server, search happens along with indexing. Until
log volumes increase beyond what one server can handle easily, this is fine. In fact,
splitting off the search instance might actually hurt performance as there is more
overhead involved in running a distributed search.
Most configurations pertaining to search are managed through the web interface.
The configuration specifically concerning distributed search is maintained at
Manager | Distributed search .
Common data sources
Your data may come from a number of sources; these can be files, network ports,
or scripts. Let's walk through a few common scenarios.
Monitoring logs on servers
In this scenario, servers write their logs to a local drive, and a forwarder process
monitors these logs. This is the typical Splunk installation.
The advantages of this approach include:
• This process is highly optimized. If the indexers are not overworked, events
are usually searchable within a few seconds.
• Slowdowns caused by network problems or indexer overload are handled
gracefully. The forwarder process will pick up where it left off when the
slowdown is resolved.
• The agent is light, typically using less than 100 megabytes of RAM and a
few percent of one CPU. These values go up with the amount of new data
written and the number of files being tracked. See inputs.conf in Chapter
10 , Configuring Splunk , for details.
• Logs without a time zone specified will inherit the time zone of the machine
running the forwarder. This is almost always what you want.
• The hostname will be picked up automatically from the host. This is almost
always what you want.