Databases Reference
In-Depth Information
The overall migration approach would be as follows:
1.
Data preparation as per the JSON file format.
2.
Data extractions into flat files as per the JSON file format or
extraction of data from the processed data store using custom
data loaders.
3.
Data loading using in-built or custom loaders into Cassandra
data structure(s).
4.
The various activities for all the different stages in migration
are further discussed in detail in below sections.
Data Preparation and Extraction:
ETL is the standard process for data extraction,
transformation and loading (Figure
6-24
). At the end of the ETL process, reconciliation
forms an important function. This includes validation of data with the business
processes. The ETL process also involves the validation and enrichment of the data
before loading into staging tables.
Figure 6-24.
Schema migration data preparation and data extraction process
The following activities are executed during data preparation:
1.
Creation of database objects: Necessary staging tables are
to be created as per the requirements based on which will
resemble standard open interface / base table structure.
2.
Validate and transform data before load from the given source
(dumps/flat files).
