Database Reference
In-Depth Information
intermediate table can be scaled by 2 to 3 orders of magnitude without
noticeably impacting query latency, allowing the intermediate table to hold
more granular records.
In traditional databases, query results are usually ephemeral and special
action has to be taken to stash them in temporary tables or external stores.
BigQuery takes a different approach, making query results available for
24 hours or indefinitely if assigned a name. This section illustrated how
applications can take advantage of this behavior.
Table Snapshots
BigQuery provides table management features that make it simple to handle
many aspects of the data life cycle, including backups and snapshots of
changing datasets to enable historical analysis. Developers should, of
course, consider integrating separate backup systems when appropriate;
Chapter 12, “External Data Processing,” which covers exports, might be
useful in that regard. This section focuses on facilities available within
BigQuery.
These features are enabled by BigQuery's capability to cheaply create copies
of tables. This is achieved by running a table copy job that specifies a source
and destination table. This job usually completes in less than 1 minute
independent of the table size. Although the job is free, it is subject to the
overall job quota, and storage charges begin to accrue for both copies of the
a copy job is configured.
Now look at a concrete archiving requirement to see how this might be
utilized. In the sample application you have an AppEngine Datastore table,
devices
, in which you can record all current installations of the mobile
app. In the next section, “AppEngine Datastore Integration,” you learn how
a snapshot of this table is made available for queries in BigQuery. Because
rows in this table are continuously updated, it is useful to have access to
historical versions because you can then compare how installations vary
over time. The goal is to have a new snapshot of this table every day but
retain some number of older versions.
Listing 11.2
illustrates how you can
combine the update step with copy jobs to meet this requirement.
Listing 11.2
Backups using copy
