Adaptive Execution of Scientific Workflow Applications on Clouds - Cloud Computing with e-Science Applications - page 80

This fact motivated us to adopt a data-driven approach to understanding the
dynamics of the public transport system in Singapore. To achieve that, a scal-
able complex system modeling for a sustainable city (S 3 ) has been developed
to study how the city will behave under different planning scenarios.
The goal of S 3 is to provide insights to users on what-if scenarios for a
day-to-day public transport system by leveraging on a synthetic journey
function that generates agent-based models for public transport dynamics
simulation. This insight will provide information on the future public trans-
port infrastructure preparedness to handle the growing population and the
preparedness for emergencies in cases of breakdowns in the public trans-
port system.
Scaling areas that we address in this context are (1) the extract-transform-
load (ETL) or preprocessing that is required to train the synthetic journey
function that generates the agent-based model; (2) the agent-based generation
required to generate millions of agents that represent the increasing popula-
tion and public transport infrastructure; and (3) the large-scale agent-based
simulation that is required to handle, track, and process each of the agents
and to support complex interactions between agents to provide insight on
what-if scenarios for the public transportation system in Singapore.
We tackled the large-scale computation requirements by designing
agent-based complex system modeling supported by an adaptive cloud
WfMS [12] for workflow scheduling and handling big data and dynamic
resource scaling on public and private clouds.
The S 3 application has three phases: preprocessing, data analysis, and
agent-based simulation. Figure 4.3 shows our S 3 application architecture,
which comprises an adaptive cloud WfMS, ETL or preprocessing algorithm,
data analysis algorithm, and agent-based simulation.
ETL or preprocessing . The synthetic data set for the application is
based on the studies of trends and random sampling of daily public
commuters' activities in Singapore. It consists of 1-second time gran-
ularities for 7 days' duration with approximately 3 million journeys
per day. Based on the synthetic data set, we extract and transform the
data for travel duration for each origin-station to destinations-station
(OD-pair) of 90 x 90 by three different route choices. The order of
complexity in this phase is O( n 2 ), where n represents the number
of stations.
Data analysis . The objective of this phase is to understand commuter
demand and, based on data analysis results, create or improve the
journey function of all possible OD pairs, possible routes for each
OD pair, and temporal travel demand. The order of complexity in
this phase is also O( n 2 ), where n represents the number of stations.