sparklyr 0.7

We are excited to share that sparklyr 0.7 is now available on CRAN! Sparklyr provides an R interface to Apache Spark. It supports dplyr syntax for working with Spark DataFrames and exposes the full range of machine learning algorithms available in Spark. Features in this release:

  • Adds support for ML Pipelines which provide a uniform set of high-level APIs to help create, tune, and deploy machine learning pipelines at scale.

  • Enhances Machine Learning capabilities by supporting the full range of ML algorithms and feature transformers.

  • Improves Data Serialization, specifically by adding support for date columns.

  • Adds support for YARN cluster mode connections.

The full blog post is available in the RStudio Blog site: