Databases Reference
In-Depth Information
FIGURE 10.10
Semantic Framework.
data, including text, image, video, and audio data. Processing data and presenting data for visualiza-
tion at both ends requires a more robust architecture, which is the semantic framework.
Figure 10.10
shows the concept of the semantic framework architecture. The framework consists
of multiple layers of processing and data integration techniques that will be deployed as a part of the
next-generation data warehouse. The layers and their functions include the following.
Lexical processing
This layer can be applied to both input data processing of Big Data and the processing of data explo-
ration queries from the visualization layer. Lexical processing includes processing tokens and streams
of text. The three main subcomponents of lexical processing include:
●
Entity extraction
—a process to identify key data tokens that can include keys and master data
elements. For example, identifying “product_name” from a twitter feed.
●
Taxonomy
—a process to navigate across domains within the text stream to identify the contexts in
the text that may be feasible, and discover relationship attributes for cross-hierarchy navigation.
●
Relationship models
—a process to derive the relationship between different data elements
and layers resulting in an exploration roadmap. This process will use outputs from the prior
components in this process.
Clustering
In this process all the data from lexical processing will be clustered to create a logical grouping of
data processed in the Big Data layers and from any data exploration queries. The subcomponents in
this layer include:
