# Spark MLib From MLlib is a scalable machine learning library built on top of Spark. The machine learning library is composed of two distinct packages, which are \[17:03]: 1. `org.apache.spark.mllib` : RDD-based library of some common machine learning algorithms. This package **will be deprecated in future releases.** 2. `org.apache.spark.ml` : Library of machine learning algorithms that leverages datasets and data frames structures. The package supports tasks pipeline and stages that are described and illustrated in the next section. MLlib consists of two packages that leverage different core data structures. The package `org.apache.spark.ml` maintains an interface for use with Spark DataFrames. This package also maintains a high-level interface for building machine learning pipelines that help standardize the way in which you perform the above steps. The lower-level package, `org.apache.spark.mllib`, maintains interfaces for Spark’s low-level, RDD APIs. This book will focus exclusively on the DataFrame API. The RDD API is the lower-level interface, which is in maintenance mode (meaning it will only receive bug fixes, not new features) at this time. It has also been covered fairly extensively in other books on Spark and is therefore omitted here. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://nag-9-s.gitbook.io/machine-learning/mlib.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.