External Data Processing - Google BigQuery Analytics - page 480

Running Extract Jobs

By this point, you should understand how the BigQuery Jobs API works; if
not, you should read the section titled “Jobs” in Chapter 5. Since we assume
you're familiar with starting jobs and waiting for them to complete, the code
to do so is not shown here in the text. The job management code used in
the subsequent listings is encapsulated in the JobRunner object defined
in job_runner.py file. This file is included within the supplemental
download material for this chapter.

Listing 12.2 runs a BigQuery Extract job via a JobRunner and then
downloads the results using a GcsReader , which was defined in the
previous listing.

Listing 12.2 : Exporting a table via an Extract job

(extract_and_read.py)

import json
import logging
import sys
import time

# Imports from local files in this directory:
import auth
from gcs_reader import GcsReader
from job_runner import JobRunner

def make_extract_config(source_project_id,
source_dataset_id,

source_table_id,

destination_uris):
'''Creates a dict containing an export job
configuration.'''

source_table_ref = {
'projectId': source_project_id,
'datasetId': source_dataset_id,
'tableId': source_table_id}
extract_config = {
'sourceTable': source_table_ref,