Database Reference
In-Depth Information
Online model evaluation
Combining machine learning with Spark Streaming has many potential applications and
use cases, including keeping a model or set of models up to date on new training data as it
arrives, thus enabling them to adapt quickly to changing situations or contexts.
Another useful application is to track and compare the performance of multiple models in
an online manner and, possibly, also perform model selection in real time so that the best
performing model is always used to generate predictions on live data.
This can be used to do real-time "A/B testing" of models, or combined with more advanced
online selection and learning techniques, such as Bayesian update approaches and bandit
algorithms. It can also be used simply to monitor model performance in real time, thus be-
ing able to respond or adapt if performance degrades for some reason.
In this section, we will walk through a simple extension to our streaming regression ex-
ample. In this example, we will compare the evolving error rate of two models with differ-
ent parameters as they see more and more data in our input stream.
