Database Reference
In-Depth Information
'expirationTime': long(time.time() * 1000 +
EXPIRATION_MS)
},
**daily).execute()
This code ends up retaining daily versions of the table for the last 30 days,
by virtue of the expiration time set on the daily snapshots, and quarterly
versions indefinitely.
Figure 11.1
illustrates how the data is moved between
the daily, current, and quarterly tables. This aligns well with classic
versioning schemes that retain a larger number of recent snapshots and
older snapshots at a coarser resolution. This enables analyses at higher
resolution over recent time intervals and lower resolution analyses over
longer intervals.
Figure 11.1
Multiresolution backups
A backup strategy serves as a defense against data loss, but also as a
safeguard against inadvertent data corruption due to bugs in software or
operator error. Cloud storage has an excellent track record for data
durability because providers have a lot of experience building redundant
fault-tolerant systems. However, these measures do not help with a bug that
corrupts the stored data. This is another scenario in which the table copy
feature can be used effectively to prevent such corruption from occurring,
especially in the management of tables that are being periodically updated
by appending new data. One way to do this safely is to load the new data
into a staging table. Then run a suite of verification queries against the new
