Database Reference
In-Depth Information
Tools that come with Hadoop support the -conf option, but it's straightforward to make
your programs (such as programs that run MapReduce jobs) support it, too, using the
Tool interface.
GenericOptionsParser, Tool, and ToolRunner
Hadoop comes with a few helper classes for making it easier to run jobs from the com-
mand line. GenericOptionsParser is a class that interprets common Hadoop
command-line options and sets them on a Configuration object for your application
to use as desired. You don't usually use GenericOptionsParser directly, as it's
more convenient to implement the Tool interface and run your application with the
ToolRunner , which uses GenericOptionsParser internally:
public interface Tool extends Configurable {
int run ( String [] args ) throws Exception ;
}
Example 6-4 shows a very simple implementation of Tool that prints the keys and values
of all the properties in the Tool 's Configuration object.
Example 6-4. An example Tool implementation for printing the properties in a Configura-
tion
public class ConfigurationPrinter extends Configured implements Tool {
static {
Configuration . addDefaultResource ( "hdfs-default.xml" );
Configuration . addDefaultResource ( "hdfs-site.xml" );
Configuration . addDefaultResource ( "yarn-default.xml" );
Configuration . addDefaultResource ( "yarn-site.xml" );
Configuration . addDefaultResource ( "mapred-default.xml" );
Configuration . addDefaultResource ( "mapred-site.xml" );
}
@Override
public int run ( String [] args ) throws Exception {
Configuration conf = getConf ();
for ( Entry < String , String > entry: conf ) {
System . out . printf ( "%s=%s\n" , entry . getKey (), entry . getValue ());
}
return 0 ;
}
public static void main ( String [] args ) throws Exception {
int exitCode = ToolRunner . run ( new ConfigurationPrinter (), args );
Search WWH ::




Custom Search