Database Reference
In-Depth Information
Offering the ability to split the flow of work into a series of parallel streams, the
fork
and
join
control nodes are
used as a pair. For instance, you can use these to run several Pig data-processing scripts in parallel. The
join
control
node will not complete until all of the
fork
actions have completed.
<fork name="pig-fork">
<path start="pig-manufacturer"/>
<path start="pig-model"/>
</fork>
Optional in the workflow, the
kill
control node stops the workflow. It is useful for error conditions; if any actions
are still running when an error occurs, they will be ended.
<kill name="fail">
<message>Workflow died, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
The
decision
control node uses a
switch
statement with a series of cases and a default option to decide which
control flow to use. The first case to be True is used; otherwise, the default case is used. In the
pig-decision switch
statement the workflow uses a file system test of file size and will pass control to the end control node if the data
required by the workflow is not greater than 1 GB.
<decision name="pig-decision">
<switch>
<case to="pig-fork">
${fs:fileSize("${hdfsRawData}) gt 1 * GB}
</case>
<default to="end"/>
</switch>
</decision>
Oozie Workflow Actions
The best way to understand an Oozie workflow action is to examine an example. I present the following Pig
Latin-based action to define the values of the JobTracker and name node. The
prepare
section deletes data for the
action. The
configuration
section then defines the Pig action queue name, and it is followed by a
script
section
that specifies the script to call. Finally, the
OK
and
error
options define which nodes to move to, depending on the
outcome of the Pig script.
<action name="pig-manufacturer">
<pig>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<prepare>
<delete path="${hdfsEntityData}/manufacturer"/>
</prepare>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
Search WWH ::
Custom Search