Database Reference
In-Depth Information
127.0.0.1:8020 failed on local exception: java.io.IOException:
javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException:
No valid credentials provided
(Mechanism level: Failed to find any Kerberos tgt)]
The operation fails because we don't have a Kerberos ticket. We can get one by authentic-
ating to the KDC, using
kinit
:
%
kinit
Password for hadoop-user@LOCALDOMAIN:
password
%
hadoop fs -put quangle.txt .
%
hadoop fs -stat %n quangle.txt
quangle.txt
And we see that the file is successfully written to HDFS. Notice that even though we car-
ried out two filesystem commands, we only needed to call
kinit
once, since the Kerber-
os ticket is valid for 10 hours (use the
klist
command to see the expiry time of your
tickets and
kdestroy
to invalidate your tickets). After we get a ticket, everything works
just as it normally would.
Delegation Tokens
In a distributed system such as HDFS or MapReduce, there are many client-server interac-
tions, each of which must be authenticated. For example, an HDFS read operation will in-
volve multiple calls to the namenode and calls to one or more datanodes. Instead of using
the three-step Kerberos ticket exchange protocol to authenticate each call, which would
present a high load on the KDC on a busy cluster, Hadoop uses
delegation tokens
to allow
later authenticated access without having to contact the KDC again. Delegation tokens are
created and used transparently by Hadoop on behalf of users, so there's no action you
need to take as a user beyond using
kinit
to sign in, but it's useful to have a basic idea
of how they are used.
A delegation token is generated by the server (the namenode, in this case) and can be
thought of as a shared secret between the client and the server. On the first RPC call to the
namenode, the client has no delegation token, so it uses Kerberos to authenticate. As a
part of the response, it gets a delegation token from the namenode. In subsequent calls it
presents the delegation token, which the namenode can verify (since it generated it using a
secret key), and hence the client is authenticated to the server.
When it wants to perform operations on HDFS blocks, the client uses a special kind of
delegation token, called a
block access token
, that the namenode passes to the client in re-
sponse to a metadata request. The client uses the block access token to authenticate itself
