Database Reference
In-Depth Information
Chapter 2. A New World for Time
Series Databases
As we saw with the old ship's logs described in
Chapter 1
, time series data—tracking events
or repeated measurements as a function of time—is an old idea, but one that's now an old
idea in a new world. One big change is a much larger scale for traditional types of data. Dif-
ferences in the way global business and transportation are done, as well as the appearance of
new sources of data, have worked together to explode the volume of data being generated.
It's not uncommon to have to deal with petabytes of data, even when carrying out traditional
types of analysis and reporting. As a result, it has become harder to do the same things you
used to do.
In addition to keeping up with traditional activities, you may also find yourself exposed to
the lure of finding new insights through novel ways of doing data exploration and analytics,
some of which need to use unstructured or semi-structured formats. One cause of the explo-
sion in the availability of time series data is the widespread increase in reporting from
sensors. You have no doubt heard the term Internet of Things (IoT), which refers to a prolif-
eration of sensor data resulting in wide arrays of machines that report back to servers or com-
municate directly with each other. This mass of data offers great potential value if it is ex-
plored in clever ways.
How can you keep up with what you normally do and plus expand into new insights? Work-
ing with time series data is obviously less laborious today than it was for oceanographer
Maury and his colleagues in the 19th century. It's astounding to think that they did by hand
the painstaking work required to collect and analyze a daunting amount of data in order pro-
duce accurate charts for recommended shipping routes. Just having access to modern com-
puters, however, isn't enough to solve the problems posed by today's world of time series
data. Looking back 10 years, the amount of data that was once collected in 10 minutes for
some very active systems is now generated every second. These new challenges need differ-
ent tools and approaches.
The good news is that emerging solutions based on distributed computing technologies mean
that now you can not only handle traditional tasks in spite of the onslaught of increasing
levels of data, but you also can afford to expand the scale and scope of what you do. These
