Database Reference
In-Depth Information
check whether those cookie identities were correctly running on the back
end through the bidding system. And finally, when we bid on them, whether
were we actually getting some impressions.
This means a detective game or puzzle, right? You expect to see about a hun-
dred thousand impressions. You find that the system only sees three impres-
sions, and you start wondering what the hell happened to them on the way
through. To figure this out, you need to pull very specific data sets out of a
large Hadoop cluster and trace the data, like specific events, as it goes through
the system, and see whether they come out on the other side. And if they do
not come out, you have to figure out where they got lost. That is the detective
work in action, just figuring out what is wrong with the data.
Gutierrez:
What about your routine tasks?
Perlich:
In regards to my more routine tasks, almost every week I rebuild
and study our inventory models. This is a basic process done with a couple
of semi-cron jobs that I run. The inventory models are run in the back end
every time there is a bid request. Remember, we get as many as 30 billion bid
requests per day. So these models are predictive models that estimate how
good the current URL for the current bid request is for a particular campaign.
If it is good, then we bid on the request. Otherwise, we let it pass.
As an example, with our travel campaigns, we know that if we get a bid request
from Kayak, travel campaigns convert with a factor of 4. So what happens is
that for Kayak URLs and the travel campaigns, there is a factor in the system
that multiplies the bid price by 4, or by 3.5, or by 4.7. So depending on what
the model is estimating, these models will decide whether or not to bid and
at what level.
So my routine is to basically start up and run them. When they are done,
I quickly eyeball the results. This process is not fully automated because the
technology it relies on is a little bit sensitive to the data stream and bid request.
Everything that has a bid request—supply and demand in the auctions—is a
little bit tricky. I prefer to look at the results in order to make sure it passes
my personal smell test. I will probably spend some time just looking at the
results and making sure they are within the ballpark of where we wanted
them. Then I hand them over to Gabriel, the head of account management,
so that we can discuss them. The way we interface with brands, like Nike and
others, is that there is a human that manages that relationship and also the
campaigns. So they pick how many ads are delivered and for what goals.
To some extent, we give recommendations on which of the many models is
the best, but they have some override because models optimize to one thing.
Sometimes the campaign is measuring something else or has a different attri-
bution metric, so there is a little bit of a translation. There is a human step
