Databases Reference
In-Depth Information
Let us revisit the metadata subject area first and then understand how it integrates into the world
of Big Data. Sources of metadata include:
●
Metadata generated automatically for data and business rules
●
Metadata created by the designer of the systems
●
Metadata procured from third-party sources
There are fundamentally nine types of metadata that are useful for information technology and data
management across the enterprise from transaction processing to analytical and reporting platforms.
Technical metadata
Technical metadata consists of metadata that is associated with data transformation rules, data storage
structures, semantic layers, and interface layers.
●
Metadata for data model and physical database includes length of a field, the shape of a data
structure, the name of a table, the physical characteristics of a field, the number of bytes in a table,
the indexes on a table, and data definition language (DDL) for a table.
●
Business processing metadata includes information about:
●
The system of record for a specific piece of data.
●
Transformations that were performed on which source data to produce data in the data
warehouse/datamart.
●
Tables and columns used in the particular process in the data warehouse/datamart and what the
transformations mean.
●
Administrative metadata.
Business metadata
Business metadata refers to the data describing the content available in the data warehouse/datamart,
including:
●
The structure of the data.
●
The values that are stored within the attributes.
●
Create and update date and time.
●
Processing specifics if any for the field or attribute.
Contextual metadata
Contextual metadata is data that sets the “context” of your data. This is more related to processing
large objects like text, images, and videos. Examples of contextual metadata include newsfeeds, tags
on images, etc. Contextual metadata is the hardest to collect.
Process design-level metadata
Process design-level metadata consists of the following:
●
Source table
●
Target table
