Information Technology Reference
In-Depth Information
testsarerunagainst alltheupgradedservers.Thesetestsareusuallyverysimplistic, gener-
ally just verifying that the code is not crashing and live queries are succeeding.
Iftroubleisfound(i.e.,ifthecanarydies),theprocessisstopped.Atthispointtheserv-
ers that were upgraded can be rolled back. Alternatively, if there is enough capacity, they
can be shut down until a new release becomes available.
Canarying is not a testing process. The canary process is a method for deploying a re-
leaseintoproductionthatdetectsbadpushesandpreventsthemfrombeingvisibletousers.
The main difference between a testing process and a canary process is that it is acceptable
for the test process to fail. If a testing process fails, you've prevented a bad release from
reaching live users. More pedantically, the testing process has succeeded in detecting a de-
fective release. That is a good thing.
Conversely, you don't want a canary process to fail. A failed canary means something
was missed by the testing process. A failed canary should be so rare that it is cause to stop
development anddedicate resourcestodetermining whatwentwrongandwhichadditional
testing needs to be added to prevent this failure in the future. Only then can new roll-outs
begin. Canarying is an insurance policy against accidental bad releases, not a way to detect
bad releases.
Testing and canarying are often conflated, but should not be. What some people call
canarying is really testing new releases on live users. Testing should be done in a testing
environment, not on live users.
Search WWH ::
Custom Search