Databases Reference
In-Depth Information
• You need to either define all IP addresses on the same network segment, or use
network bridging.
• Changing the IP address requires root access to the system.
• Sometimes you need to update address resolution protocol (ARP) caches. Some
network devices might cache ARP entries for too long, and might not instantly
switch an IP address to a different MAC address. We've seen lots of cases where
network hardware or other components decide not to cooperate, and thus the
various parts of the system don't agree on where the IP address really lives.
• You need to make sure the network hardware supports fast IP takeover. Some
hardware requires MAC address cloning for this to work properly.
• It's possible for a server to keep its IP address even though it's not fully functional,
so you might need to physically shut it down or disconnect it from the network.
This is known by the lovely acronym of STONITH: “shoot the other node in the
head.” It's also called “fencing,” which is a more delicate and official-sounding
name.
Floating IP addresses and IP takeover can work well for failover between machines that
are local to each other—that is, on the same subnet. In the end, however, you need to
be aware that this isn't always a bulletproof strategy, depending on your network hard-
ware and so on.
Waiting for Changes to Propagate
Often, when you define redundancy on one layer, you have to wait for a lower layer to
actually carry out a change. Earlier in this chapter, we pointed out that changing servers
through DNS is a weak solution because DNS is slow to propagate changes. Changing
IP addresses gives you more control, but IP addresses on a LAN also depend on a lower
layer—ARP—to propagate changes.
Middleman Solutions
You can use proxies, port forwarding, network address translation (NAT), and hard-
ware load balancers for failover and failback. They're nice because unlike other solu-
tions that tend to introduce uncertainty (do all of the system components really agree
on which one is the master database? can it be changed instantaneously and atomi-
cally?), they're a central authority that controls connections between the application
and the database. However, they do introduce a single point of failure themselves, and
you'll need to make them redundant to avoid that problem.
One of the nice things you can do with such a solution is make a remote data center
appear to be on the same network as your application. This lets you use techniques
such as floating IP addresses to make your application begin communicating with an
entirely different data center. You can configure each application server in each data
