Database Reference
In-Depth Information
Real-Time Optimization
If a metric is being monitored for changes, presumably there is some action
thatcanbetakentomitigateundesirablechangesinthemetricoremphasize
positive changes in the metric. Otherwise, outside of its entertainment
value, there is little use in tracking the metric.
The more interesting situations that arise are when the metric is an outcome
that can be affected by a process that can be automatically controlled. For
example, an e-commerce website might have two different designs that are
hypothesized to have different returns (either the number of sales or the
dollars per session for each design). Testing which design performs best is
known as A/B testing and is often performed sequentially by launching a
new site and tracking the average session value compared to the old site.
A more sophisticated approach, often used by large consumer websites, is
to perform the experiment by exposing some fraction of users to the new
design and tracking the same value.
However, if the size of the exposed populations for each design can be
controlled, then it is possible to automatically choose the best design over
time without explicit control. One way of doing this is using the so-called
multi-armed bandit optimization strategy.
The premise of this strategy is that a player has entered a casino full of slot
machines (also known as one-armed bandits). Each machine has a different
rate of payout, but the only way to determine these rates is to spend money
and play the machine. The challenge for the player is then to maximize
their return on the investment (or if it is a real casino, minimize its loss).
Intuitively, the player would begin by assuming that all machines are equal
and playing all of them equally. If some machines have higher payouts than
the rest, the player would then begin to focus more of their attention on that
subset of machines, eventually abandoning all other machines.
In the context of website optimization, the two different designs are the
one-armed bandits, and “playing” the game assigns a visitor to one of the
twodesignswhentheyarriveatthesite.Theonlyproblemistodecidewhich
design a visitor should see.
For the purposes of demonstration, assume that a visitor either buys a
product or not, and all of the products have the same price. In this case, it is
only necessary to model the probability of a purchase rather than modeling
the values directly. As each user arrives, the exposed design is tracked, as is
