Database Reference
In-Depth Information
repl: replication data too stale, halting
Fri Jan 28 14:19:27 [replsecondary] caught SyncException
Recall that the oplog is a capped collection. This means that entries in the collection
eventually age out. Once a given secondary fails to find the point at which it's synced
in the primary's oplog, there's no longer any way of ensuring that the secondary is a
perfect replica of the primary. Because the only remedy for halted replication is a
complete resync of the primary's data, you'll want to strive to avoid this state. To do
that, you'll need to monitor secondary delay, and you'll need to have a large enough
oplog for your write volume. You'll learn more about monitoring in chapter 10.
Choosing the right oplog size is what we'll cover next.
S
IZING
THE
REPLICATION
OPLOG
The oplog is a capped collection and as such, it can't be resized once it's been created
(at least, as of MongoDB v2.0).
6
This makes it important to choose an initial oplog size
carefully.
The default oplog sizes vary somewhat. On 32-bit systems, the oplog will default to
50 MB, whereas on 64-bit systems, the oplog will be the larger of 1 GB or 5% of free
disk space.
7
For many deployments, 5% of free disk space will be more than enough.
One way to think about an oplog of this size is to recognize that once it overwrites
itself 20 times, the disk will likely be full.
That said, the default size won't be ideal for all applications. If you know that your
application will have a high write volume, you should do some empirical testing
before deploying. Set up replication and then write to the primary at the rate you'll
have in production. You'll want to hammer the server in this way for at least an hour.
Once done, connect to any replica set member and get the current replication info:
db.getReplicationInfo()
Once you know how much oplog you're generating per hour, you can then decide
how much oplog space to allocate. You should probably shoot for being able to with-
stand at least eight hours of secondary downtime. You want to avoid having to com-
pletely resync any node, and increasing the oplog size will buy you time in the event of
network failures and the like.
If you want to change the default oplog size, you must do so the first time you start
each member node using
mongod
's
--oplogSize
option. The value is in megabytes.
Thus you can start
mongod
with a 1
GB
oplog like so:
mongod --replSet myapp --oplogSize 1024
H
EARTBEAT
AND
FAILOVER
The replica set heartbeat facilitates election and failover. By default, each replica set
member pings all the other members every two seconds. In this way, the system can
6
The option to increase the size of a capped collection is a planned feature. See
https://jira.mongodb.org/
browse/SERVER-1864
.
7
Unless you're running on OS X, in which case the oplog will be 192 MB. This smaller size is due to the assump-
tion that OSX machines are development machines.
