Database Reference
In-Depth Information
Fortheseapplications,themoneybeingspentindifferentenvironmentsand
on different sites is being managed on a per-minute basis in much the same
way as the stock market. In addition, these buys are often being managed to
some sort of metric, such as the number of purchases (called a conversion)
or even the simpler metric of the number of clicks on an ad. When a visitor
arrives at a website via a modern advertising exchange, a call is made to a
number of bidding agencies (perhaps 30 or 40 at a time), who place bids on
thepageviewinrealtime.Anauctionisrun,andtheadvertisementfromthe
winning party is displayed. This usually happens while the rest of the page
is loading; the elapsed time is less than about 100 milliseconds. If the page
contains several advertisements, as many of them do, an auction is often
being run for all of them, sometimes on several different exchanges.
All the parties in this process—the exchange, the bidding agent, the
advertiser, and the publisher—are collecting this data in real time for
various purposes. For the exchange, this data is a key part of the billing
process as well as important for real-time feedback mechanisms that are
used for a variety of purposes. Examples include monitoring the exchange
for fraudulent traffic and other risk management activities, such as
throttling the access to impressions to various parties.
Advertisers, publishers, and bidding agents on both sides of the exchange
are also collecting the data in real time. Their goal is the management and
optimization of the campaigns they are currently running. From selecting
thebid (in thecase oftheadvertiser) orthe“reserve” price (in thecase ofthe
publisher), to deciding which exchange offers the best prices for a particular
type of traffic, the data is all being managed on a moment-to-moment basis.
A good-sized advertising campaign or a large website could easily see page
views in the tens or hundreds of millions. Including other events such as
clicks and conversions could easily double that. A bidding agent is usually
acting on the part of many different advertisers or publishers at once and
will often be collecting on the order of hundreds of millions to billions of
events per day. Even a medium-sized exchange, sitting as a central party,
can have billions of events per day. All of this data is being collected, moved,
analyzed, and stored as it happens.
Social Media
Another newer source of massive collections of data are social media
sources, especially public ones such as Twitter. As of the middle of 2013,
