Database Reference
In-Depth Information
the other nodes, but will never catch up to those nodes in the total amount
of data stored on it unless you balance the stored data. To balance the stored
data,youcanrunthefollowingcommandfromaHadoopcommandprompt,
hadoop balancer -threshold N
where
N
is the percentage you want the nodes to be within each other. For
example, ifyouwantallthenodestobewithin 5%ofeachotherforanactual
potential of each node being as much as 10% apart in total data stored, run
the following:
hadoop balancer -threshold 5
Once you've created a solution that will benefit your business it's time to
hand that solution off to your operations team so that it can operate the
daily jobs, respond to user requests, and plan for future growth. In the next
section we'll discuss some of the work necessary to make that transition to
the operations team effective.
Going Forward
After deploying your solution, you need to continue to monitor its
performance and plan for its growth. You learn much more about what to
monitor for in Chapter 16, “Operational Big Data Management.” For now,
we'll limit our coverage to handing the solution off to operations and what
needs to happen post deployment.
The Handoff to Operations
The hand off to operations should be thoroughly thought about, discussed,
and planned for a long time before the day comes to actually do it. In fact,
the handoff to operations should be planned for from day 1 of the project.
Questions you should be thinking about include the following:
• Does anyone in operations understand the big data paradigm?
• Does anyone in operations have experience with Hadoop?
• Who or what group is going to operate the solution?
• If the cluster fails at night, how do we notify and respond?
• If a job fails at night, how do we respond?
