Databases Reference
In-Depth Information
...
int myproperty = job.getInt("myjob.myproperty", 0);
if (myproperty < 0) {
System.err.println("Invalid myjob.myproperty: " + myproperty);
System.exit(0);
}
JobClient.runJob(job);
return 0;
}
When you allow the user to specify custom properties, it's good practice for the driver
to validate any user input. The example above ensures that the user will not be allowed
to specify a negative value for myjob.myproperty .
7.2
Probing for task-specific information
In addition to retrieving custom properties and global configuration, we can also use
the getter methods
on the configuration object to obtain certain state information
about the current task and job. For example, in the Mapper you can grab the map.
input.file property to get the file path to the current map task. This is exactly what
the configure() method in the datajoin package's DataJoinMapperBase does to
infer a tag for the data source.
this.inputFile = job.get("map.input.file");
this.inputTag = generateInputTag(this.inputFile);
Table 7.1 lists some of the other task-specific state information.
Table 7.1 Task-specific state information one can get in the configuration object
Property
Type
Description
String
The job ID
mapred.job.id
mapred.jar
String
The jar location in job directory
job.local.dir
String
The job's local scratch space
mapred.tip.id
String
The task ID
mapred.task.id
String
The task attempt ID
mapred.task.is.map
boolean
Flag denoting whether this is a map task
int
The ID of the task within the job
mapred.task.partition
map.input.file
String
The file path that the mapper is reading from
long
The offset into the file of the start of the current
mapper's input split
map.input.start
map.input.length
long
The number of bytes in the current
mapper's input split
mapred.work.output.dir String
The task's working (i.e., temporary) output directory
 
Search WWH ::




Custom Search