Creating KPIs - The Practice of Cloud System Administration - page 395

The team could rationalize any of these end points to be the end time. Thus, we go back
to Step 1 and use what we have learned to do better.
19.3.2 The Second Pass
Our original definition was a very narrow definition of the end result. Let's broaden the
definition and focus on what the user sees as the desired result.
To the user, the end result is that the VM is ready for use for the purpose the user inten-
ded.
Ideally, the user always receives the VM that has been requested: it has the right name,
size(amountofRAM,disk,andvCPU),operatingsystem,andsoon.Thus,ouridealworld
implies that the right information is gathered from the user and the user receives a VM that
works as requested.
Perhaps some roadblocks might prevent the customer from using the machine right
away. These should be included in the metric. For example, to use the machine requires
DNS changes to be propagated, access controls to be implemented, and so on.
Also consider the use of company resources. Can users request an infinite number of
VMs? Who pays for them? Billing for resources is often an effective way to lead users to
restrain their use of a resource. We'll use that.
Therefore we revise our ideal-world definition as follows: Users get a usable VM, as
they requested, as quickly as possible, with billing arrangements established.
Continuing our example into Step 2, we define the start time as when the request is ini-
tially received from the user. The end time is when the VM is usable by the requester. We
can define “usable” as the user being able to log into the machine. This automatically in-
cludes DNS propagation and access control as well as issues we are unaware of.
The draft KPI becomes:
The 90th percentile creation time, which starts when the request is created, and ends
when the user is able to log into the machine.
We use a percentile instead of an average because, as discussed in Section 17.5 , aver-
ages can be misleading. A single long-delayed creation would unfairly ruin an average.
An ideal world has no resource shortages. Anytime a new VM is needed, there would
be capacity to allocate one. New capacity would come online just in time to fulfill any re-
quest. We could add a metric related to capacity planning but it is better to keep the KPI
at a high level of abstraction, rather than micro-managing how the service is run. We are
concerned with the end result. If capacity planning is not done right, that fact will surface
due to the KPI we have constructed as requests are held while waiting for new capacity to