Database Reference
In-Depth Information
For these use cases BigQuery does support generating large results, but it
requires that the
allowLargeResults
flag be explicitly set to
true
in the
query job configuration. If you are wondering why this flag does not default
to
true
, the reason is that large result generation uses a different execution
mode that is slower than the default mode that has the size restriction.
Queries that generate small results are orders of magnitude more common
than large result queries, so it would be a shame to slow all of them down
by defaulting to the large result execution path. Because you will need to
reference the results in queries or export jobs, you are required to supply a
destination table to save the results when the flag is set to true. Note that,
unlike with anonymous tables, you will begin to accrue storage costs for your
query results when the query completes, and it will not be automatically
garbage collected.
job['configuration'] = {
'query': {
'allowLargeResults': True,
'destinationTable': {
'projectId': 'myproject',
'datasetId': 'mydataset',
'tableId': 'my_big_result'
}
'query': 'SELECT . . .'
}
}
You can also enable the flag when issuing queries using the command-line
client.
$
bq query --allow_large_results \
--
destination_table foo.bar \
'SELECT * FROM [some.table] WHERE . . .'
This feature is powerful and should certainly be utilized when appropriate,
but it is not a good idea to always enable it because then all your queries will
be slower.
