Information Technology Reference
In-Depth Information
decide whether today is a good day for a code push. For example, you might have a policy
of not doing pushes on critical days such as when the finance department is closing the
topics and outages would be especially bad. You might notice an unusually large number
ofbrokenbuildslately,whichleadsyoutorealizethedevelopersmaynotbehavingagood
week.Thismightmakeyouwaryofacceptingareleasewithoutfirsthavingachatwiththe
lead developers to see if they've been under a lot of stress. These issues seem like reason-
able things to consider.
For some people, these hunches may be an excuse to avoid continuous deployment. In
reality, hunches provide a roadmap for how to embrace it. There is a bit of truth to all such
hunches, so why not turn them into measurable things that can automatically disable auto-
matic deployments?
Here is a list of factors that should be taken into consideration when deciding whether
to pause continuous delivery. Some of them may initially sound like things only humans
can determine. However, with a little creativity, they can all be automated and used to gate
automated code pushes:
•
Build Health:
A recent bout of build failures often indicates other stability issues.
Automatic pushes can be blocked if
n
of the last
m
builds failed. This can serve as
a proxy for measuring developer stress or rushed work.
•
Test Comprehensiveness:
Only highly tested code should be automatically
pushed. There are metrics such as “percentage of code coverage” that can be used
as indicators that testing has become lax.
•
Test Reproducibility:
Flakey tests are tests that sometimes pass and sometimes
fail. These can be detected by running the tests multiple times and in random order.
Delivery can be paused automatically if test reproducibility is low.
•
Production Health:
There is no single test that indicates a system is healthy, but a
good proxy is that the monitoring system has no outstanding alerts.
•
Schedule Permission:
Automated pushes are forbidden if there is a scheduled
“change freeze” for holidays, quarterly financial reporting, or days when too many
key people are on vacation. A simple list of dates to pause deployments covers
these situations.
•
Oncall Schedule:
To avoid dragging the current oncall engineer out of bed, pushes
may be paused during typical sleeping hours. If there are multiple oncall teams,
each covering different times of the day, each may have a different idea of waking
hours. Even a three-shift, follow-the-sun arrangement may have hours when no
one is entirely awake.
Search WWH ::
Custom Search