Assessments - The Practice of Cloud System Administration - page 440

• Building the software is still done manually.
• Testing is done manually.
• Some unit tests exist.
• Source files are kept in version-controlled repositories.
• Software packages are used as the means of delivering the end result.
• If multiple platforms are supported, each is repeatable, though possibly independ-
ently.
Level 3: Defined
• Building the software is automated.
• Triggers for automated builds are defined.
• Expectations around unit test coverage are defined; they are less than 100 percent.
• Metrics for bug lead time, code lead time, and patch lead time are defined.
• Inputs and outputs of each step are defined.
Level 4: Managed
• Success/fail build ratios are measured and tracked on a dashboard.
• Metrics for bug lead time, code lead time, and patch lead time are collected auto-
matically.
• Metrics are presented on a dashboard.
• Unit test coverage is measured and tracked.
Level 5: Optimizing
• Metrics are used to select optimization projects.
• Attempts to optimize the process involve collecting before and after metrics.
• Each developer can perform the end-to-end build process in his or her own sand-
box before committing changes to a centralized repository.
• Insufficient unit test code coverage stops production.
• If multiple platforms are supported, building for one is as easy as building for them
all.
• The software delivery platform is used for building infrastructure as well as applic-
ations.