Database Reference
In-Depth Information
An alternative to this approach, and one that overcomes this limitation,
is WebHDFS. WebHDFS is an HTTP-based REST (Representational State
Transfer) API that fully implements the same file system commands found
in the FS shell.
Accessing this API is accomplished by embedding commands into HTTP
URL requests and taking advantage of the standard HTTP operations (
GET
,
POST
,
PUT
,
DELETE
). To better illustrate, consider the following example to
open a file via WebHDFS:
http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=OPEN
To deconstruct this URL for better understanding, note the following:
• The
<HOST>
and
<PORT>
arguments point to the location of the
NameNode.
• The
<PATH>
argument is the file being requested.
• The
op
querystring parameter passes in the file system operation.
An HTTP
GET
request is issued using the example URL. After the path and
location metadata is queried from the NameNode, an HTTP 307 temporary
redirect is returned to the requestor, as shown here:
HTTP/1.1 307 TEMPORARY_REDIRECT
Location: http://<DATANODE>:<PORT>/webhdfs/v1/
<PATH>?op=OPEN…
Content-Length: 0
TheredirectcontainstheactualpathtotheDataNodethathoststhedatafile
or block. The client can then follow the redirect to stream the data directly
from the source.
As previously mentioned, the WebHDFS offers a complete implementation
of the FS shell commands,
Table 4.2
list some of the file system commands
and their WebHDFS equivalents.
