Monitoring Fundamentals - The Practice of Cloud System Administration - page 335

as the product changes—for example, if a new release requires significantly different re-
sources. CP is the topic of Chapter 18 .
Product Management (PM) requires very low-resolution data for calculating key per-
formance indicators (KPIs) such as conversion rates, counts of users, and analysis of user
retention (often called 7-day actives or 30-day actives). PM benefits from large amounts of
historic data for long-term views and visualizations to help understand trends.
Missing (R, L, D) Combinations
A keen observer will notice that not every combination of R, L, and D appears in
the model. Some do describe other operations-related functions. (R+, L+, D-) is
what a load balancer requires to determine if it should send traffic to a particular
backend.(R+,L-,D-)coversthekindofloganalysisthatisdoneinperiodicallyin
batches. This leaves (R-, L+, D-) and (R-, L+, D+), which we have not observed
in the wild...yet.
16.3 What to Monitor
What should be monitored is different for every organization. The general strategy is to
start with the business's KPIs and collect related measurements.
The following are some example KPIs:
• Availability: Have a web site that is up 99.99 percent of the time (measured from
outside the datacenter).
• Latency: The 90th percentile latency for the homepage should not exceed 400 ms
(request to render time).
• Urgent Bug Count: There should be no more than n outstanding Severity-0 bugs.
• Urgent Bug Resolution: All Severity-0 bugs should be closed within 48 hours.
• Major Bug Resolution: All Severity-1 bugs should be closed within 10 days.
• Backend Server Stability: There should be no more than n percent queries return-
ing HTTP 5xx Server Error.
• User Satisfaction: There should be no more than n percent abandoned carts.
• Cart Size: The median number of items in a shopping card per order should be n .
• Finance: A total of n revenue this month.
If you do not know your organization's KPIs, stop right now and find out what they are. If
there aren't any, prepare your resume because your company is a rudderless ship. Alternat-
ively, you can declare yourself the captain and invent the KPIs yourself.