R interface for MLeap

Install mleap

mleap can be installed from CRAN via

install.packages("mleap")

or, for the latest development version from GitHub, using

devtools::install_github("rstudio/mleap")

Setup

Once mleap has been installed, we can install the external dependencies using:

library(mleap)
install_mleap()

Another dependency of mleap is Maven. If it is already installed, just point mleap to its location:

options(maven.home = "path/to/maven")

If Maven is not yet installed, which is the most likely case, use the following to install it:

install_maven()

Create an MLeap Bundle

1. Start Spark session using sparklyr

   library(sparklyr)
   sc <- spark_connect(master = "local", version = "2.2.0")
   mtcars_tbl <- sdf_copy_to(sc, mtcars, overwrite = TRUE)

2. Create a fit an ML Pipeline

   pipeline <- ml_pipeline(sc) %>%
     ft_binarizer("hp", "big_hp", threshold = 100) %>%
     ft_vector_assembler(c("big_hp", "wt", "qsec"), "features") %>%
     ml_gbt_regressor(label_col = "mpg")
   
   pipeline_model <- ml_fit(pipeline, mtcars_tbl)

3. A transformed data frame with the appropriate schema is required for exporting the Pipeline model

   transformed_tbl <- ml_transform(pipeline_model, mtcars_tbl)

4. Export the model using the ml_write_bundle() function from mleap

   model_path <- file.path(tempdir(), "mtcars_model.zip")
   ml_write_bundle(pipeline_model, transformed_tbl, model_path)

   ## Model successfully exported.

5. Close Spark session

   spark_disconnect(sc)

At this point, we can share mtcars_model.zip with the deployment/implementation engineers, and they would be able to embed the model in another application. See the MLeap docs for details.

Test the mleap bundle

The mleap package also provides R functions for testing that the saved models behave as expected. Here we load the previously saved model:

model <- mleap_load_bundle(model_path)
model

## MLeap Transformer
## <db23a9f1-7b3d-4d27-9eb0-8675125ab3a5> 
##   Name: pipeline_fe6b8cb0028f 
##   Format: json 
##   MLeap Version: 0.10.0-SNAPSHOT

To retrieve the schema associated with the model use the mleap_model_schema() function

mleap_model_schema(model)

## # A tibble: 6 x 4
##   name       type   nullable dimension
##   <chr>      <chr>  <lgl>    <chr>    
## 1 qsec       double TRUE     <NA>     
## 2 hp         double FALSE    <NA>     
## 3 wt         double TRUE     <NA>     
## 4 big_hp     double FALSE    <NA>     
## 5 features   double TRUE     (3)      
## 6 prediction double FALSE    <NA>

Then, we create a new data frame to be scored, and make predictions using the model:

newdata <- tibble::tribble(
  ~qsec, ~hp, ~wt,
  16.2,  101, 2.68,
  18.1,  99,  3.08
)

# Transform the data frame
transformed_df <- mleap_transform(model, newdata)
dplyr::glimpse(transformed_df)

## Observations: 2
## Variables: 6
## $ qsec       <dbl> 16.2, 18.1
## $ hp         <dbl> 101, 99
## $ wt         <dbl> 2.68, 3.08
## $ big_hp     <dbl> 1, 0
## $ features   <list> [[[1, 2.68, 16.2], [3]], [[0, 3.08, 18.1], [3]]]
## $ prediction <dbl> 21.06529, 22.36667