Database Reference
In-Depth Information
Concentrate on the Data Challenge, Not the Technology
I believe that the most important solution to problems caused by data silos isn't actu-
ally technological. The most important goal is to understand the scope of questions
that you aim to answer and choose the most effective solution.
Data warehousing is not always the right solution to overcoming the challenges of
data silos, and neither is a distributed processing system such as Hadoop. In fact, both
require a great deal of investment and skill to use effectively. In practice, many cross-
organizational data questions can be answered without the need for a data warehouse.
However, investing in a data warehousing solution does make sense if the same ques-
tions are being asked over and over, and regular reporting and compliance needs must
be met.
A distributed processing system combined with a fast, ad hoc analytics database
might be the best solution when faced with the need to process data quickly in the
face of changing requirements.
For security and compliance challenges, such as those posed by Sarbox, data ware-
housing can be a good solution. If data is automatically transformed and loaded into a
data warehouse that can only be directly accessed by a few, then it is easier to ensure
that the data are correct during audits. Building automated ETL processes for this pur-
pose, as well as the strict modeling of data that goes along with it, can help enforce the
stringency necessary to fulfill good compliance practices.
Empower Employees to Ask Their Own Questions
The concept of the data warehouse is often tied to hierarchical models of organiza-
tional structure. Executives, the decision makers and captains of the ship, are expected
to have a high-level view of an organization's data, and employees down the tree are
merely tasked with data collection. The organization's leaders are expected to use this
high-level information for decision making and analysis.
This view contrasts with models that aim to build in data analysis as a fundamental
aspect of the organization's structure. Industries that are being “eaten by software,” as
Marc Andreessen puts it, should be able to make decisions backed by metrics whenever
possible, whether the CEO is involved or not.
Trying to come up with a definition for “data-driven organization” is very simi-
lar to the problem of trying to define “business intelligence”—it's a simple term that
describes a goal that can be very difficult to reach in practice. For some, the con-
cept of a
data-driven organization
is an ideal state in which every single decision
is carefully prefaced by metrics. In practice, organizations that use data to become
successful seem to operate in much the same way that researchers use the scientific
method: Someone in the organization uses data to test a hypothesis from the observa-
tions and insights that come from interactions with customers and others outside the
organization.
This concept means that everyone in the organization has access to the data tools
necessary to be able to get answers to data-related questions. This doesn't mean that
Search WWH ::
Custom Search