Databases Reference
In-Depth Information
Figure 8-13.
Recommendation system conceptual architecture
This brings big data into the picture. Succeeding with data and building
new markets, or changing the existing markets is the game being played in many
high-stakes scenarios. Some companies have found a way to build their big data
recommendation/machine-learning platform, giving them the edge in bringing better
and better products even faster to the market. The more data we give to our algorithms,
the better-targeted results we get. A recommendation platform using Hadoop would have
the following components: ETL, feature generation, feature selection, recommendation
algorithms, A/B testing, serving, tracking, and reporting.
In the sections below, we go over use cases and details of solving them in the
Hadoop ecosystem. We will also specifically cover a set of machine-learning algorithms
for solving the various recommendation use cases. While Mahout fits well with Hadoop
map-reduce framework, there are also elegant ways of plugging in other non-distributed
systems/algorithms into Hadoop.
Let's first review some basic concepts related to recommendation system.
In a classical model of recommendation system, there are “users” and “items.”
A “user” has associated metadata (or content) such as age, gender, race, and other
demographic information. “Items” also has its metadata, such as text description, price,
weight, etc.
On top of that, there are interactions (or transactions) between the user and items,
such as user A downloading/purchasing item X or user A giving a rating 5 to a product Y.
In a real-world scenario, you will find many-to-many relationships between users and
products.
