Database Reference
In-Depth Information
BigQuery Reserved Capacity
BigQuery offers the ability to reserve processing capacity in a dedicated
virtual cluster in units of 5 GB processed per second. This might sound
strange, since we just said that the rough goal is to process 50 GB per
second for on-demand. Does that mean reserved capacity charges you
more for less performance?
There are a couple of things to note with respect to reservations:
• Reserved capacity gives you the ability to run queries that preempt
other users, up to your capacity limit.
• Reserved capacity gives you the optional ability to 'burst' over your
capacity rate. This means your queries can use the pool of
on-demand resources in addition to reserved resources.
• On-demand (non-reserved) capacity is best-effort only.
Performance may vary significantly from day to day, even from
query to query, based on load of the overall system. Reservations
give you the ability to be first in line for resources and to expect
more stable performance.
Query performance should continue to scale sublinearly (that is, if you
double the size, it will take less than double the time) up to at least 500 GB
of data processed in the query. So if you have a 100 MB table that takes 3
seconds to query and you increase the size a thousand times to 100 GB, it
might take only 5 seconds to query. Increasing the size of the table will allow
BigQuery to use more hardware to run the query.
There is a limit to the number of execution nodes that will be assigned to
any one query, however. Based on current cluster sizing, that limit comes at
approximately one-half a terabyte of data processed. If you start with a 1 TB
table that you can query in 20 seconds and double it to 2 TB, your queries
will now likely take 40 seconds. Note that the relevant size here just includes
the fields that are touched. If you have 100 fields but just read one of them,
the effective size is just the size of that single field.
There isn't actually a hard maximum table size you can process in BigQuery,
other than saying that after a certain point, querying the tables may take
