Creating KPIs - The Practice of Cloud System Administration - page 400

19.5 Summary
ManagingbyusingKPIsisaradicaldeparturefromtraditionalITmanagement.Itiseffect-
ive because it sets goals and permits the smart people whom you hired to work out how to
achieve them. Those people are closer to the task and more knowledgable about the tech-
nical details at hand, making them better suited to inventing ways to achieve the goal.
Creating effective KPIs is difficult—and it should be. It yields a huge return on in-
vestment when done right. Something that has a 1:1000 payoff is not going to be easy.
KPIs should be specific, measurable, achievable, relevant, and time-phrased. Poorly writ-
tenKPIshaveunintendedconsequenceswhenpeoplefollowtherulesaswrittenratherthan
what you intended. It is important to think through all the ways that the KPI may be inter-
preted, and the actions that people might take to improve their performance on this metric.
Spend time up front examining and amending the KPI to make sure that the KPI will
most likely give the result you want. Do not over-complicate the KPI. If you suspect that
the KPI might trigger an adverse behavior, measure and publish that behavior as well, and
make sure that your managers know to watch out for and discourage it.
Remember that the KPI should be achievable. One hundred percent availability is not
achievable, unless the definition of “availability” is adjusted to cover more than you inten-
ded. However four 9s (99.99 percent) is achievable. Google's Error Budget KPI success-
fully uses that target to achieve the desired result: a stable service with innovative new fea-
tures deployed frequently. It is an excellent example of how a good KPI can benefit every-
one.
Exercises
1. What is a KPI?
2. What are the SMART criteria? Briefly describe each one.
3. Give examples of unintended side effects of KPIs.
4. What are the steps for creating KPIs? Evaluate which steps are the most difficult
and provide justification as to why they are difficult.
5. Which KPIs do you track in your environment and why?
6. Create an effective KPI for assessing a service in your environment. After creating
it, have three other people tell you how they would maximize their personal gain.
Revise the KPI. Report on the ideal (Step 1), the initial KPI(s), people's reactions,
and the final KPI.
7. How does the KPI you created address each of the SMART criteria?