Database Reference
In-Depth Information
Tools that come with Hadoop support the
-conf
option, but it's straightforward to make
your programs (such as programs that run MapReduce jobs) support it, too, using the
Tool
interface.
GenericOptionsParser, Tool, and ToolRunner
Hadoop comes with a few helper classes for making it easier to run jobs from the com-
mand line.
GenericOptionsParser
is a class that interprets common Hadoop
command-line options and sets them on a
Configuration
object for your application
to use as desired. You don't usually use
GenericOptionsParser
directly, as it's
more convenient to implement the
Tool
interface and run your application with the
ToolRunner
, which uses
GenericOptionsParser
internally:
public interface
Tool
extends
Configurable
{
int
run
(
String
[]
args
)
throws
Exception
;
}
of all the properties in the
Tool
's
Configuration
object.
Example 6-4. An example Tool implementation for printing the properties in a Configura-
tion
public class
ConfigurationPrinter
extends
Configured
implements
Tool
{
static
{
Configuration
.
addDefaultResource
(
"hdfs-default.xml"
);
Configuration
.
addDefaultResource
(
"hdfs-site.xml"
);
Configuration
.
addDefaultResource
(
"yarn-default.xml"
);
Configuration
.
addDefaultResource
(
"yarn-site.xml"
);
Configuration
.
addDefaultResource
(
"mapred-default.xml"
);
Configuration
.
addDefaultResource
(
"mapred-site.xml"
);
}
@Override
public
int
run
(
String
[]
args
)
throws
Exception
{
Configuration conf
=
getConf
();
for
(
Entry
<
String
,
String
>
entry:
conf
) {
System
.
out
.
printf
(
"%s=%s\n"
,
entry
.
getKey
(),
entry
.
getValue
());
}
return
0
;
}
public static
void
main
(
String
[]
args
)
throws
Exception
{
int
exitCode
=
ToolRunner
.
run
(
new
ConfigurationPrinter
(),
args
);