Advanced Deployments - Implementing Splunk: Big Data Reporting and Development for Operational Intelligence - page 343

By default, the forwarder will use auto load balancing, specified by autoLB=true .
Essentially, the forwarder will switch between indexers on a timer. This is the only
option available for the Universal Forwarder and light forwarder.
On a heavy forwarder, the setting autoLB=false will load balance by event. This
is less efficient and can cause results to be returned in a non-deterministic manner,
since the original event order is not maintained across multiple indexers.
Understanding typical outages
With a single Splunk instance, an outage—perhaps for an operating system
upgrade—will cause events to queue on the Splunk forwarder instances. If there are
multiple indexers, forwarders will continue to send events to the remaining indexers.
Let's walk through a simplified scenario. Given these four machines, with the
forwarders configured to load balance their output across two indexers as shown
in the following figure:
While everything is running, half of the events from each forwarder data will be sent
to each indexer. If one indexer is down, we are left with only one indexer as shown
in the figure:
A few things happen in this case:
• All events will be sent to the remaining indexer.
• All events stored on our unavailable indexer will not be included in search
results. Splunk 5 can help with this problem, at the cost of extra disks.
• Queries for recent events will work because these events will be stored on the
remaining indexer, assuming the one indexer can handle the entire workload.