China Grid and Related Dependability Research - Grid Computing: Infrastructure, Service, and Applications - page 85

operation for more than 18 months, in spite of some of the servers being
up-and-down and in-and-out from time to time. At the time of writing,
realcourse is composed of 20 servers spanned over 10 major cities
distributed on CERNET with 1500 hours quality course video, which
has grown from four servers initially within the Peking University
campus. As a result, we normally observe 20-100 online users at any
time. In turn, these user activities help us to bring the service to a level
of robustness.
The 1500 hours of video are from different universities within the
ChinaGrid. They are uploaded to different servers with a permanent
backup copy created in a server other than the original one, and l ow
from server to server on demand. If one server is down for some reason,
its duty is automatically taken up by another server(s). Once it is up again,
a maintenance session will occur automatically to bring it up to a consis-
tent state, in case any transactions occurred during its down time (e.g.,
some more videos were uploaded to the system).
Realcourse is a successful application of distributed computing techno-
logies in a geographically wide area. In contrast to some traditional
distributed fault-tolerant services, realcourse emphasizes giving clients
access to the service—with reasonable response times—for as much of
the time as possible, even if some results do not conform to sequential
consistency.
Realcourse possesses some distinguished characteristics. It is aimed at
non-critical application of video on demand and i le storage in which
temporal inconsistency is acceptable only if consistency of the whole sys-
tem is i nally reached. By eliminating the “update” operation, realcourse
greatly reduces the consistency requirement. The length of exchanged
message among servers is kept constantly small and thus heavy commu-
nication trafi c is observed with the growth of server numbers.
The servers in realcourse are not equal to each other. Only two servers
are chosen to keep two permanent physical copies of i les. No effort is
made to keep track of those temporal copies of i les but the existence of
these copies greatly improves the performance of downloading opera-
tions by way of a wide-area network cache. In the new version of the
consistency-check procedure, a pervasive search is started to i nd those
“lost” copies in case both permanent copies of the i le are corrupted. The
loop topology of all servers makes it possible for each server to run a
consistency-check procedure independently without any overhead. In
fact, a more sophisticated consistency-check procedure is in develop-
ment at the time of writing. Knowledge from the area of “autonomous
computing” is of much help. By exploiting the reliable communication
provided by asynchronous messaging middleware, realcourse hides the
failures of the network, which are relatively common in our observation
of a wide-area network from servers. The consistency of servers is
eventually kept in a “delayed” fashion.