Databases Reference
In-Depth Information
6.4.6 Code Filer
All of the actual code files were stored on a separate “code filer”, which was a big
server stuffed with drives that were RAIDed together.
These files were fronted by the Lighttpd web server, which was configured to
require a time-limited token for authorized access.
6.4.7 Live Versus Stand-by Versus Dev Systems
The actual infrastructure was even more complex than what has been described
above. The code searchers, page searchers, master searcher and code filer all com-
prised a snapshot of the state of the system, for both code and data. As such, we had
three copies of these—one in production, one on standby, and one being provisioned
as the next “live” system.
6.4.8 Release Process
Whenever our “dev” system was sufficiently tested, we would do a “flip.” This con-
sisted of switching the VIPs (virtual IP addresses) for the master server and code
filer to the new system, which would then cause the web tier to start using the
new code and data. The previously live system would become the backup, which
we could easily flip back to if the new live system had serious problems. And the
previous backup system would become the new dev system, which we would start
provisioning as the next live system.
This continuous rotation of systems worked well, but at the cost of lots of extra
hardware. We also spent a lot of time pushing data around, as provisioning a new
server meant copying many terabytes of data to the searchers and the filer from our
back-end systems.
6.4.9 Post Mortem
The public search system worked, but required a lot of manual labor to build and
deploy updates to the data, and had a lot of moving pieces. This complexity resulted
in a lot of time spent keeping the system alive and happy, versus improving and
extended the search functionality.
In retrospect it would have made more sense to first focus on the Krugle En-
terprise system, and handle the public site via setting up multiple instances of these
servers, each with a slice of the total projects. Then the delta between a stock Krugle
Search WWH ::
Custom Search