Databases Reference
In-Depth Information
CHAPTER
11
Data-Driven Architecture for
Big Data
What information consumes is rather obvious: it consumes the attention of its recipients.
Hence a wealth of information creates a poverty of attention, and a need to allocate that
attention efficiently among the overabundance of information sources that might consume it.
—Herbert Simon
INTRODUCTION
Data is one of the biggest assets of any enterprise, large or small. With the right set of information,
you can make business decisions with higher levels of confidence, as you can audit and attribute the
data you used for the decision-making process. The complexities of managing data in structured envi-
ronments have been a challenge that enterprises struggle with, and things will be worse when you
look at adding Big Data into this situation. Enterprises that have paid attention to data management as
an asset will be winners when building out the next-generation data warehouse.
To goal of this chapter is to provide readers with data governance in the age of Big Data. We will
discuss the goals of what managing data means with respect to the next generation of data warehous-
ing and the roles of metadata and master data in integrating Big Data into a data warehouse.
Data management refers to the process of collecting, processing, storing, and distributing data.
The data management techniques that we have been using today in the modern database world are
based on requirements that were developed for legacy systems dating back from punch cards, to
mainframes, to analytical data processing systems.
Figure 11.1
shows the fundamental stages of data management across enterprises. There are sev-
eral cycles of activities within each stage of processing that create complexities for managing the
entire process end to end. Before we look into the details of these processes, let us take a brief look at
metadata and master data.
Metadata
Metadata
is defined as data about data or, in other words, information about data within any data envi-
ronment. The origins of metadata can be traced to library systems from many years ago, where the
classiication, tagging, and cataloging of topics provided the fundamental information classiication
and retrieval. Applied to information technology, metadata provides a natural way to integrate data and
interrogate data.
219
