Database Reference
In-Depth Information
are available immediately. If you do not stream any data to a table for an
extended duration (a couple of hours or so), this initial delay may recur.
Looking at the insert request you can see that the actual data inserted is
nested within a JSON object. The top-level object must contain a list-valued
field named
rows
. Each entry in the list is an object with a field named
json
whose value is the record being inserted. The JSON value that appears at
this field is identical to the row that you would use in JSON formatted input
to load jobs. The only difference is that new lines are not significant and
are treated the same as any other whitespace that appears between JSON
values. Here is the full specification for the insert request:
{
"kind": "bigquery#tableDataInsertAllRequest",
"rows": [
{
"insertId":
string
,
"json": {
record data
}
}
]
}
The only additional field that was not part of the example request is
insertId
. This field is actually important and requires the most
explanation. Just as job ID protects you from inserting the same job
multiple times, the insert ID prevents the same record from being inserted
multiple times if a request is retried. Because the client might receive a
failure response even if the server has successfully handled the request, the
client might attempt to reinsert a record that has already been received by
the service. If you set the
insertId
to the same value across multiple insert
requests, the service uses it to de-duplicate records. You should always
insert the same record for a given ID because the service does not guarantee
which record (first, last, or any in between) will be saved for a given ID.
But there is an important caveat: This deduplication works only for a finite
amount of time. The exact duration may change but it is currently around
5 minutes. Beyond that, the service discards the state associated with the
insertId
. After this time the service treats an insert with the same ID as
a new distinct record. In most scenarios in which a client has to retry an
operation, this time window is sufficient to avoid duplication. However, it is
