Operational Excellence - The Practice of Cloud System Administration - page 407

Level 3: Defined
At this level, the roles and responsibilities of the process are defined and confirmed. At the
previous level, we learned what needed to be done. At this level, we know who is respons-
ible for doing it, and we have definitions of how to measure correctness. The correctness
measurements might not be collected, but at least the people involved know what they are.
Each step has a series of checks to find errors as they happen rather than waiting until the
end of the process to find out what went wrong. Duplication of effort is minimized. If we
need to increase the rate of production, we can duplicate the process or add more capacity.
Level 4: Managed
At this level, aspects of the process are measured. How long each step takes is measured,
including what portion of that time involved waiting to begin the process itself. Measure-
mentsincludehowoftentheprocessisdoneeachmonth,howoftenitisperformedwithout
error, and how many times an exceptional case arises that requires special treatment and
ad hoc processes. Variation is measured. These measurements are collected automatically.
There is a dashboard that shows all of these measurements. It is easy to spot bottlenecks.
Postmortems arepublishedwithinaspecific amountoftimeafteranexception. Exceptions
are cataloged and periodically reviewed. Requests to change a process are justified using
measurement data to show that there is a problem. Capacity needs are predicted ahead of
time.
Level 5: Optimizing
At this level, the measurements and metrics are being used to optimize the entire system.
Improvements have been made at all the previous levels but the improvements made here
are prompted by measurements and, after the change is made, the success is evaluated
based on new measurements. Analyzing the duration of each step exposes bottlenecks and
delays. Measurements indicate month-over-month improvements. We can stress-test the
system to see what breaks, fix it, and then run the system at the new level.