Game Development Reference
In-Depth Information
dependencies that originate inside and crossing this area's borders. If none of the
inputs have changed, the area can be safely tagged as fully processed.
Knowing that the number of dependencies can be as large as several million,
pruning areas can provide big savings both in the size of the data sets and the
number of tasks that will ultimately be executed. The advantages will become
evident after knowing the details of the proxying techniques of Section 1.6.3.
1.5.5 Dump the Dependency Graph
Once the dependency graph has been built, dump it to a file! Later builds can read
it and use the contents as a starting point. An updated DAG can be built at a
reduced cost, allowing faster start-up times. Moreover, many implicit dependencies
will be ready, with a considerable reduction of the number of scanner executions.
The chances of performing conceptually simple but technically dicult tasks, such
as giving an estimate of a build time, are also improved. If none of the task
templates have been modified, any subsequent builds will only need to regenerate
the areas of the graph whose inputs have changed. “What has changed” can be
redefined based on the expectations of the current build: a quick, incremental build
focused on a single level, a full build, replacement of a few objects, etc.
Another important consequence is the fact that graph generation and task
scheduling are now decoupled. This provides interesting opportunities: Tool up-
dates become more localized, and the determination and evaluation of tasks become
independent. It is even possible to use alternative schedulers optimized for quick
turnaround, make use of heavy parallelism, or rely on the facilities of certain com-
puting hosts, or use simple heuristics such as first-come, first-serve, etc.
Finally, the information contained in the dependency graph can be invaluable for
finding and studying bottlenecks, locating areas for improvement, and fine-tuning
the system. Understanding how such a complex system is performing may be a
daunting task in the absence of execution data and clear logs. Simple formats such
as plain text, xml, .doc, or GraphML and tools like Microsoft Excel and languages
supporting xquery or even the command grep can be invaluable.
1.6 Advanced Techniques
1.6.1 Caching
This method focuses on file reuse and is heavily inspired by the equivalent sys-
tems for source-code compilation. A successful example is the CCache package
[Tridgell 11].
With a caching scheme, the outputs of compilation steps are stored in a repos-
itory. During the evaluation of each node, the scheduler checks for the existence
of a cached result that corresponds to the given set of input nodes. In the event
of a cache hit, the result can be eciently retrieved. Cache misses fall back to a
