How To: Export & Import MLlib Model to S3

Description:

This article discusses how to export Spark Machine Learning Models to S3 in the JSON format.

Prerequisite:

Have a model created inside of a Spark Notebook or a Spark Job in Analyze. 

How To:

Once the model has been created the following line of code can be used to save the model as a JSON file inside of S3:

model.save(sc,"s3://...file_path.../")

The model may be imported and used in a Spark Notebook or a Spark Job in Analyze with the following lines of code:

import org.apache.spark.mllib.recommendation.{ALS, MatrixFactorizationModel, Rating};

val myModel = MatrixFactorizationModel.load(sc, "s3://...");

myModel.predict(user_id, product_id);

This same code could be executed through the Qubole API for integration with additional workflows. 

 

Have more questions? Submit a request

Comments

Powered by Zendesk