MapReduce - Hadoop: The Definitive Guide - page 77

When the argument is a file, the simplest variant returns an array of FileStatus ob-
jects of length 1. When the argument is a directory, it returns zero or more FileStatus
objects representing the files and directories contained in the directory.
Overloaded variants allow a PathFilter to be supplied to restrict the files and director-
ies to match. You will see an example of this in the section PathFilter . Finally, if you spe-
cify an array of paths, the result is a shortcut for calling the equivalent single-path
listStatus() method for each path in turn and accumulating the FileStatus ob-
ject arrays in a single array. This can be useful for building up lists of input files to pro-
cess from distinct parts of the filesystem tree. Example 3-6 is a simple demonstration of
this idea. Note the use of stat2Paths() in Hadoop's FileUtil for turning an array
of FileStatus objects into an array of Path objects.
Example 3-6. Showing the file statuses for a collection of paths in a Hadoop filesystem
public class ListStatus {
public static void main ( String [] args ) throws Exception {
String uri = args [ 0 ];
Configuration conf = new Configuration ();
FileSystem fs = FileSystem . get ( URI . create ( uri ), conf );
Path [] paths = new Path [ args . length ];
for ( int i = 0 ; i < paths . length ; i ++) {
paths [ i ] = new Path ( args [ i ]);
}
FileStatus [] status = fs . listStatus ( paths );
Path [] listedPaths = FileUtil . stat2Paths ( status );
for ( Path p : listedPaths ) {
System . out . println ( p );
}
}
}
We can use this program to find the union of directory listings for a collection of paths:
% hadoop ListStatus hdfs://localhost/ hdfs://localhost/user/tom
hdfs://localhost/user
hdfs://localhost/user/tom/books
hdfs://localhost/user/tom/quangle.txt