Databases Reference
In-Depth Information
the operational systems. Here, real-time data warehousing refers to the how
quickly data is made available through the data warehousing environment.
The question at hand is what real time means to you. The objective is to get
data to the business at the right time. The right time means getting the data to
the business users in time to help them make decisions. This may mean real
time in some cases, but in many others it means ''by the start of business the
next morning.'' The goal is to get the data to the right people when it will be
most useful for decision making.
Many organizations agonize over building a real-time environment, but
their business does not operate at that speed. For insurance companies, for
example, it may seem interesting to have real-time policy and claims data
in order to watch loss ratios unfold. However, unless you are in the excess
and surplus business, you can't change your rates immediately anyway, so
if you see a pattern emerge this afternoon, you would still need to build a
case and present it to the state board of insurance before you could institute
a rate change. In other words, if a business decision can't be implemented
immediately anyway, what is the value of building the infrastructure needed
to support loading real-time data? In this case, the right time may be to
continue with daily loads of data to monitor the business.
In some cases, the data may need to be refreshed every hour or perhaps every
fifteen minutes. If data is truly needed up to the second, then a continuous
trickle of data may be more appropriate. In any case, the infrastructure must be
built to support these increased demands. The data may be loaded into an
oper-
ational data store (ODS)
, an integrated, up-to-the-moment operational database
or into a partition of the data warehouse set aside for this high-performance
real-time data loading. At the end of the day, if the solution that has been
devised is working, then great! If your organization does not have a real need
for time-critical data, then don't worry about these issues. Continue to load
and maintain the data with the timing that is necessary to support the speed
of your decision-making processes. There are many considerations regarding
the design, development, implementation, support, and cost of real-time or
near real-time data warehousing, a subject beyond the scope of this topic.
Unstructured Data
In years past, it was ludicrous to think of searching through textual data in a
timely manner to find useful tidbits. Think about searching the Internet, which
offers an overwhelming volume of information, in many different formats.
Therefore, we know it is possible. What does this mean for data warehousing?
Most data warehousing technology and methods have been focused on
structured data — how to organize and use it. However, many other types
of information are useful as well. Organizations often have maps, photos,
contracts, and schematics and/or blueprints. For example, it would be helpful
