Game Development Reference
In-Depth Information
the sync. We called this tool SyncBot, and it would eventually evolve to become
our final solution to working remotely.
The first incarnation of SyncBot was a simple C# command line application
that I put together in a couple days. That might have been the end of it, except
for something peculiar I noticed in the output logs. Whenever a sync restarted,
I noticed that many files that Perforce had seemingly finished were being synced
again. To be sure that it wasn't something I had caused, I verified that the same
behavior was present in the Perforce client. Further investigation revealed that
Perforce does this to prevent overwriting an existing file until the new file has been
fully synced.
This was important because many of the files that we were syncing were huge,
well over a hundred megabytes. Because the sync process was so slow, needlessly
syncing hundreds of megabytes of data a second time seemed wasteful. I was
determined to fix that.
15.5 A Parallel Approach
My plan to fix the redundancy was to sync every file individually, which would
guarantee that the server would recognize it as finished. I accomplished this by
modifying SyncBot to first run a sync preview command, then iterate through the
list of files returned, syncing them one at a time.
As I was making that change, though, I couldn't help but think about how the
data would be easy to divide up among several threads. Several past colleagues
had claimed that syncing different folders in separate instances of P4Win increased
bandwidth utilization, so I decided that it was worth investigating.
It ended up being a fairly simple task. I had already done the work of generating
the list of files using a sync preview command, so I just needed a way to distribute
those files among several worker threads. The solution I settled on was to treat
the list as a queue and let the worker threads grab files from the queue until it was
empty.
To be honest, I didn't expect to see much benefit from adding multithreading.
I assumed that we were already getting as much bandwidth from the server as
possible. However, after using this new version for several nights, it seemed like my
syncs were completing much faster than before.
I wanted to be sure that I wasn't just imagining things, so I compared the
network utilization of SyncBot and P4Win in the Task Manager. It wasn't my
imagination: SyncBot was getting more bandwidth from the server! Excited by
this new development, I timed several syncs while varying the number of threads
used.
Figure15.2
showstheseresults.
The secret to SyncBot's increased bandwidth is in the multiple connections
that it establishes with the Perforce server. Many networks cap bandwidth on a
per-connection basis, but SyncBot conveniently circumvents this limitation.
