Database Reference
In-Depth Information
COMPATIBILITY
When moving from one release to another, you need to think about the upgrade steps that are needed.
There are several aspects to consider: API compatibility, data compatibility, and wire compatibility.
API compatibility concerns the contract between user code and the published Hadoop APIs, such as the
Java MapReduce APIs. Major releases (e.g., from 1.x.y to 2.0.0) are allowed to break API compatibility,
so user programs may need to be modified and recompiled. Minor releases (e.g., from 1.0.x to 1.1.0) and
point releases (e.g., from 1.0.1 to 1.0.2) should not break compatibility.
NOTE
Hadoop uses a classification scheme for API elements to denote their stability.
The preceding rules for API compatibility cover those elements that are
marked
InterfaceStability.Stable
. Some elements of the public
Hadoop APIs, however, are marked with the
InterfaceStabil-
ity.Evolving
or
InterfaceStability.Unstable
annotations (all
these annotations are in the
org.apache.hadoop.classification
package), which mean they are allowed to break compatibility on minor and
point releases, respectively.
Data compatibility concerns persistent data and metadata formats, such as the format in which the HDFS
namenode stores its persistent data. The formats can change across minor or major releases, but the
change is transparent to users because the upgrade will automatically migrate the data. There may be
some restrictions about upgrade paths, and these are covered in the release notes. For example, it may be
necessary to upgrade via an intermediate release rather than upgrading directly to the later final release in
one step.
Wire compatibility concerns the interoperability between clients and servers via wire protocols such as
RPC and HTTP. The rule for wire compatibility is that the client must have the same major release num-
ber as the server, but may differ in its minor or point release number (e.g., client version 2.0.2 will work
with server 2.0.1 or 2.1.0, but not necessarily with server 3.0.0).
NOTE
This rule for wire compatibility differs from earlier versions of Hadoop, where
internal clients (like datanodes) had to be upgraded in lockstep with servers.
The fact that internal client and server versions can be mixed allows Hadoop 2
to support rolling upgrades.
The full set of compatibility rules that Hadoop adheres to are documented at the
Apache Software
Upgrading a cluster when the filesystem layout has not changed is fairly straightforward:
install the new version of Hadoop on the cluster (and on clients at the same time), shut
