Database Reference
In-Depth Information
down the old daemons, update the configuration files, and then start up the new daemons
and switch clients to use the new libraries. This process is reversible, so rolling back an
upgrade is also straightforward.
After every successful upgrade, you should perform a couple of final cleanup steps:
1. Remove the old installation and configuration files from the cluster.
2. Fix any deprecation warnings in your code and configuration.
Upgrades are where Hadoop cluster management tools like Cloudera Manager and
Apache Ambari come into their own. They simplify the upgrade process and also make it
easy to do rolling upgrades, where nodes are upgraded in batches (or one at a time for
master nodes), so that clients don't experience service interruptions.
HDFS data and metadata upgrades
If you use the procedure just described to upgrade to a new version of HDFS and it ex-
pects a different layout version, then the namenode will refuse to run. A message like the
following will appear in its log:
File system image contains an old layout version -16.
An upgrade to version -18 is required.
Please restart NameNode with -upgrade option.
The most reliable way of finding out whether you need to upgrade the filesystem is by
performing a trial on a test cluster.
An upgrade of HDFS makes a copy of the previous version's metadata and data. Doing an
upgrade does not double the storage requirements of the cluster, as the datanodes use hard
links to keep two references (for the current and previous version) to the same block of
data. This design makes it straightforward to roll back to the previous version of the
filesystem, if you need to. You should understand that any changes made to the data on
the upgraded system will be lost after the rollback completes, however.
You can keep only the previous version of the filesystem, which means you can't roll back
several versions. Therefore, to carry out another upgrade to HDFS data and metadata, you
will need to delete the previous version, a process called
finalizing the upgrade
. Once an
upgrade is finalized, there is no procedure for rolling back to a previous version.
In general, you can skip releases when upgrading, but in some cases, you may have to go
through intermediate releases. The release notes make it clear when this is required.
You should only attempt to upgrade a healthy filesystem. Before running the upgrade, do
a full
fsck
(see
Filesystem check (fsck)
)
. As an extra precaution, you can keep a copy of
