Operations in a Distributed World - The Practice of Cloud System Administration - page 159

Resource deallocation can cover many aspects. There may be DNS entries to be re-
moved, machines to power off, database connections to be disabled, and so on. Usually
there are complex dependencies involved. Often nothing can begin until the last user is off
the service; certain resources cannot be deallocated before others, and so on. For example,
typically a DNS entry is not removed until the machine is no longer in use. Network con-
nectionsmustremaininplaceifdeallocatingotherservicesdependsonnetworkconnectiv-
ity.
Resource disposal includes securely erasing disks and other media and disposing of all
hardware. The hardware may be repurposed, sold, or scrapped.
If decommissioning is done incorrectly or items are missed, resources will remain al-
located. A checklist, that is added to over time, will help assure decommissioning is done
completely and the tasks are done in the right order.
7.3 Organizing Strategy for Operational Teams
An operational team needs to get work done. Therefore teams need a strategy that assures
that all incoming work is received, scheduled, and completed. Broadly speaking, there are
threesourcesofoperational workandtheseworkitemsfallintothreecategories. Tounder-
stand how to best organize a team, first you must understand these sources and categories.
The three sources of work are life-cycle management, interacting with stake-holders,
and process improvement and automation. Life-cycle management is the operational work
involvedinrunningtheservice.Interactingwithstakeholdersreferstobothmaintainingthe
relationship with people who use and depend on the service, and prioritizing and fulfilling
theirrequests.Processimprovementandautomationisworkinspiredbythebusinessdesire
for continuous improvement.
No matter the source, this work tends to fall into one of these three broad categories:
• Emergency Issues: Outages, and issues that indicate a pending outage that can be
prevented, and emergency requests from other teams. Usually initiated by an alert
sent by the monitoring system via SMS or pager. ( Chapter 14 )
• Normal Requests: Process work (repeatable processes that have not yet been
automated), non-urgent trouble reports, informational questions, and initial con-
sulting that results in larger projects. Usually initiated by a request ticket system.
( Section 14.1.3 )
• Project Work: Small and large projects that evolve the system. Managed with
whatever project management style the team selects. ( Section 12.4.2 )
To assure that all sources and categories of work receive attention, we recommend this
simple organizing principle: people should always be working onprojects, with exceptions