Database Reference
In-Depth Information
defined both in a parameter file and on the command line, the last value on the command
line takes precedence.
Dynamic parameters
For parameters that are supplied using the
-param
option, it is easy to make the value
dynamic by running a command or script. Many Unix shells support command substitu-
tion for a command enclosed in backticks, and we can use this to make the output direct-
ory date-based:
%
pig -param input=/user/tom/input/ncdc/micro-tab/sample.txt \
>
-param output=/tmp/`date "+%Y-%m-%d"`/out \
>
ch16-pig/src/main/pig/max_temp_param.pig
Pig also supports backticks in parameter files by executing the enclosed command in a
shell and using the shell output as the substituted value. If the command or script exits
with a nonzero exit status, then the error message is reported and execution halts. Back-
tick support in parameter files is a useful feature; it means that parameters can be defined
in the same way in a file or on the command line.
Parameter substitution processing
Parameter substitution occurs as a preprocessing step before the script is run. You can see
the substitutions that the preprocessor made by executing Pig with the
-dryrun
option.
In dry run mode, Pig performs parameter substitution (and macro expansion) and gener-
ates a copy of the original script with substituted values, but does not execute the script.
You can inspect the generated script and check that the substitutions look sane (because
they are dynamically generated, for example) before running it in normal mode.
