Database Reference
In-Depth Information
will override the number of reducers set on the cluster or set in any client-side configura-
tion files.
The other options that
GenericOptionsParser
and
ToolRunner
support are listed
in
Table 6-1
. You can find more on Hadoop's configuration API in
The Configuration
API
.
WARNING
Do not confuse setting Hadoop properties using the
-D
property=value
option to
GenericOp-
tionsParser
(and
ToolRunner
) with setting JVM system properties using the
-D
prop-
erty
=
value
option to the
java
command. The syntax for JVM system properties does not allow any
whitespace between the
D
and the property name, whereas
GenericOptionsParser
does allow
whitespace.
JVM system properties are retrieved from the
java.lang.System
class, but Hadoop properties are
accessible only from a
Configuration
object. So, the following command will print nothing, even
though the
color
system property has been set (via
HADOOP_OPTS
), because the
System
class is not
used by
ConfigurationPrinter
:
%
HADOOP_OPTS='-Dcolor=yellow' \
hadoop ConfigurationPrinter | grep color
If you want to be able to set configuration through system properties, you need to mirror the system
properties of interest in the configuration file. See
Variable Expansion
for further discussion.
