Database Reference
In-Depth Information
Sometimes, we like to refer to all those logs and the trace data that's gener-
ated by the operation of your IT solutions as
data exhaust
. Enterprises have
lots of data exhaust, and like any pollutant, it gets discarded because it's
viewed as waste. Log data is often too strongly associated with high storage
costs. However, logs contain a great deal of potential insight, not only into
what's happening to your servers now, but what
will
happen.
Take a moment to consider how your organization does infrastructure
budgeting. You'd like to think that it's fact-based, but is it really? Do you have
a clear view of system utilization across the end-to-end platforms, including
trending? Do you understand the impact of seasonality and other events
across all groups and departments? We were able to show one of our customers
that they were planning new server acquisitions based on peak load
volumes that could be handled by existing idle systems. We helped them to
save several million dollars, resulting in a triple-digit ROI in the first year.
We helped another customer set up a central log clearinghouse rather than
letting each subgroup roll their own solution. This customer was storing logs
in a two-week window, then flushing them to avoid the cost of storage. Depart-
ments and application development teams that depended on these systems,
knowing that the logs were going to be flushed, would grab them and place
the logs on expensive SANs. There was no corporate-wide retention policy
or consistency in how the logs were being used by the various departments.
It's ironic when we think about it. These logs were being flushed because
they incurred storage expense, yet they ended up being placed, in triplicate
or more, on expensive SANs without a planned purging mechanism. This
customer used BigInsights to create a centralized log clearinghouse, and real-
ized over one million dollars in savings. Specifically, they arbitraged BigIn-
sights storage for the SAN and implemented a nine-month rolling retention
policy; after this time period, there was little value in retaining the logs. They
didn't stop there, however. Now that the logs were in one place and for a
reasonable amount of time, they were able to determine holistic trends and
issues; in short, they were able to connect the dots.
But there's more to this use case than detecting problems. This client is
now compiling a body of knowledge so that they can better anticipate failures
and understand the interaction between failures. Their service bureau can
generate best-practice remediation steps for specific problems, or tune the in-
frastructure to eliminate them. This is about discoverable preventive mainte-
nance. Some of our large insurance and retail clients need to know the
