Database Reference
In-Depth Information
08:07 AM Starting deployment.
08:07 AM Checking if deployment succeeded.
08:07 AM Will check again in 1 seconds.
…
08:20 AM Will check again in 60 seconds.
08:21 AM Checking if deployment succeeded.
08:21 AM Deployment successful.
08:21 AM Checking if updated app version is serving.
08:21 AM Completed update of app: …, version: 1
Basic AppEngine MapReduce
The AppEngine MapReduce framework enables you to leverage AppEngine
scalability to solve your data processing challenges. The authoritative
documentation
for
the
feature
is
available
at
We will port our simple script to AppEngine MapReduce to show how it
works with simple data inputs. Rather than use local files as inputs and
outputs to the script, the AppEngine version directly reads the GCS input
file and writes the output straight back to GCS.
In this example, GCS is used as the cloud equivalent of a local filesystem. We
are postponing BigQuery integration a little longer so that you can focus on
the difference between scripts intended to be executed on a single machine
and the MapReduce version that can be scaled up.
To set up your AppEngine project to run MapReduce jobs, you need to
install the AppEngine MapReduce SDK. The samples for this chapter
include an AppEngine project with the SDK installed. Instructions for
downloading the SDK into your application directory are available at the
link previously given. In addition, you need to install the library for
AppEngine/GCS
integration,
which
is
available
at
After adding this client library to your application, you need to grant
permission to the AppEngine application service account. This will let it
