Database Reference
In-Depth Information
something is going wrong. An HTTP 404 is always Not Found, so you can
handle a 404 from BigQuery the same way you do from another HTTP API,
like Google Cloud Storage.
Asynchronous Job Execution
The last important piece of BigQuery is the asynchronous job manager.
Everything that BigQuery does on your behalf that might take time, might
run into errors, or might cost a variable amount of money, runs in the
context of a job. All queries that you run are jobs, for example.
Most other query frameworks force you into a synchronous model, where
you start the query and wait for the response, which contains the query
results. When running queries synchronously, if you hit a network error or
the request times out, you have to retry the query from scratch.
BigQuery jobs are asynchronous; that is, you start a job and then check on
the progress of the job until it is done. All jobs have a unique name, either
provided by the user or generated by BigQuery. This name can be used
both to determine the status of the job and to read the results. Moreover,
because you have a name for the job, you don't have to read all the results
in one request; you can request one page at a time. Paging through results is
important because BigQuery queries can return terabytes of data.
Although asynchronous interfaces can be slightly more difficult to use, they
can be much more flexible and resistant to errors. For instance, if you use
AppEngine, all requests must complete within 60 seconds or AppEngine
kills them. If you used a synchronous model, you would have no way of
running a query that took longer than AppEngine allowed. When running
asynchronously, however, you can start the query and then poll until it is
done, which can all be done in shorter requests that don't bump up against
AppEngine time limits.
What BigQuery Isn't
We think that BigQuery is an extremely versatile and useful tool. Like any
tool, however, there are some jobs where it is a better fit than others.
And then there are some jobs where it is no help at all. BigQuery is not a
replacement for health insurance, nor should you consult it for legal advice.
If that comes as a surprise, we should probably also tell you that BigQuery is
not likely to provide psychiatric services either.
