Database Reference
In-Depth Information
My proposed extension for this is to archive snapshot views of data.
Suppose,forexample, thatauserqueries asensitive table.Howdoyoutrack
what the user saw at the time he saw it? One possible solution is to intercept
the
SELECT
statement and instead prepend it with a CETAS statement, thus
exporting the data into Hadoop creating a formal record of the data viewed.
This data could then be read from that file with a modified
SELECT
against
the external table, thus guaranteeing that the same data is read as was
recorded against theuseraccessing thedata. Theseaudit snapshots couldbe
implemented in the database itself, but this can sometimes lead to a lot of
data being generated and held in a relational store with little requirement to
actually do anything with it. This is an attempted low-cost solution to that
problem.
In summary, the advantages of this approach are as follows:
• Simple audit process
• Use of low-cost storage
• Reduced complexity for the database
• Removes storage capacity from the database
• Online retrieval if ever required
The primary disadvantage here is the network round-trip.
Data Subsetting and Obfuscation
Time for a variation on a theme: when you need to move data from one
environment to another, it is often not a simple case of backup and restore.
Consider moving data from production down to development or quality
assurance environments. Often you don't want to move all the data, just
a meaningful subset for representative testing volumes. More importantly,
you want to ensure that no personally identifiable information is exposed
in less-secure environments. Therefore, we need a way to both create
obfuscated but meaningful subsets of production data.
One option is to leverage the export capabilities of Polybase to perform a
high-performance data subsetting and obfuscation routine. Once exported,
this data couldthen bepicked upbyother PDWs oreven other systems from
Hadoop. If required, one could also run additional algorithms over the data
in Hadoop to provide further obfuscation to the data.
