Designing for Operations - The Practice of Cloud System Administration - page 38

Case Study: Google Maps Local Business Listing
Googlesubscribestolistings ofbusinessesfromvarious“local businessdirectory”
services for use in its map-related products. These information providers periodic-
ally send data that must be processed and imported into the map system. The qual-
ity of the data is disheartening: listings are often incorrect, mangled, or somehow
useless. Corrections are reported to the provider but could take months to appear
in the data feed. Some sources provide data that has good quality for certain states
and countries but categorically bad data for others. Therefore the system that im-
ports this data has a whitelist, a blacklist, and an augmentation file.
Thewhitelistindicateswhichregionstoinclude.Itisusedwhentheinformation
from a particular provider might be of high quality only for certain regions. Once
the quality of data for a particular region is verified, it is added to the whitelist. If
the quality drops, the data for that region is removed.
The blacklist identifies known bad records. It includes records that have been
identified as bad or incorrect in past batches.
The data from the business directories is augmented by data that Google pro-
duces independently of the source. This includes information sourced by Google
itself, overrides for known bad data that Google has independently corrected, and
“Easter eggs” (jokes to be included in the listings).
2.1.13 Monitoring
Operations requires visibility into how the system is working. Therefore each component
of a system must expose metrics to the monitoring system. These metrics are used to mon-
itor availability and performance, for capacity planning, and as part of troubleshooting.
Chapters 16 and 17 cover this topic in greater detail.
2.1.14 Auditing
Logging,permissions,androleaccountsaresetuptoenabletheservicetobeexaminedfor,
and pass, security and compliance audits. This area is changing rapidly, so it is always best
toconsultyourlegaldepartmentforinformationaboutthelatestlaws.Thebiggestconcerns
for corporations that are considering using public cloud services revolve around compli-
ance with the relevant laws governing their business: if they choose to use a public cloud
service, will they fail their next audit, and subsequently face massive fines or be prevented
from conducting business until they pass another audit?