Database Reference
In-Depth Information
Chapter 11
Managing Data Stored in BigQuery
The previous chapters cover how BigQuery simplifies analytics over large
datasets. BigQuery also has features to simplify data management and the
integration of analytics into an application. This chapter covers those features
and how to handle common data warehousing tasks using them.
Query Caching
As discussed in Chapter 7, “Running Queries,” BigQuery has an auto-caching
feature that enables it to reuse results across identical queries. This feature
is convenient because it is transparent to the user but is limited to instances
in which the service can guarantee that existing results from a prior query
job are identical to the results that would be generated by running the query
again, which we will elaborate on below. The application developer, on the
other hand, knows a great deal more about the use case. So when the
application can trade freshness for execution cost, it is possible to further
reduce query costs by directly managing caching. With many data
warehousing systems, it is necessary to utilize a separate caching framework,
for example Memcached, to reduce load on the query engine or the latency
of operations in a front end. With BigQuery it is usually feasible to avoid a
separate caching framework for query results by leveraging the feature that
query results are actually new tables that can be assigned an explicit name.
Different parts of the application can interact with the same query result by
accessing the appropriate named table. Next you see how this works.
Result Caching
To understand how named query results are useful, look at how to build
a page in your application that renders the top 100 most active Android
applications over, for example, the last 6 hours. Imagine something like a
leaderboard for applications, which could perhaps serve as a navigational
element leading to information about a specific application. First, you need
to formulate the query that can generate the information that needs to be
displayed:
