Database Reference
In-Depth Information
Chapter 4. Building a Recommendation
Engine with Spark
Now that you have learned the basics of data processing and feature extraction, we will
move on to explore individual machine learning models in detail, starting with recommend-
ation engines.
Recommendation engines are probably among the best types of machine learning model
known to the general public. Even if people do not know exactly what a recommendation
engine is, they have most likely experienced one through the use of popular websites such
as Amazon, Netflix, YouTube, Twitter, LinkedIn, and Facebook. Recommendations are a
core part of all these businesses, and in some cases, they drive significant percentages of
their revenue.
The idea behind recommendation engines is to predict what people might like and to un-
cover relationships between items to aid in the discovery process (in this way, it is similar
and, in fact, often complementary to search engines, which also play a role in discovery).
However, unlike search engines, recommendation engines try to present people with relev-
ant content that they did not necessarily search for or that they might not even have heard
of.
Typically, a recommendation engine tries to model the connections between users and some
type of item. In our MovieStream scenario from
Chapter 2
,
Designing a Machine Learning
System
, for example, we could use a recommendation engine to show our users movies that
they might enjoy. If we can do this well, we could keep our users engaged using our ser-
vice, which is good for both our users and us. Similarly, if we can do a good job of showing
our users movies related to a given movie, we could aid in discovery and navigation on our
site, again improving our users' experience, engagement, and the relevance of our content
to them.
However, recommendation engines are not limited to movies, books, or products. The tech-
niques we will explore in this chapter can be applied to just about any user-to-item relation-
ship as well as user-to-user connections, such as those found on social networks, allowing
us to make recommendations such as people you may know or who to follow.
Recommendation engines are most effective in two general scenarios (which are not mutu-
ally exclusive). They are explained here:
