Database Reference
In-Depth Information
As of Spark Version 1.2.0, a new, experimental API for MLlib has been released under the
ml
package (whereas the current library resides under the
mllib
package). This new API
aims to enhance the APIs and interfaces for models as well as feature extraction and trans-
formation so as to make it easier to build pipelines that chain together steps that include
feature extraction, normalization, dataset transformations, model training, and cross-valid-
ation.
In the upcoming chapters, we will only cover the existing, more developed MLlib API,
since the new API is still experimental and may be subject to major changes in the next
few Spark releases. Over time, the various feature-processing techniques and models that
we will cover will simply be ported to the new API; however, the core concepts and most
underlying code will remain largely unchanged.
