How To: Install Sparklyr package on a Cluster

Sparklyr package provides an interface between R and Apache Spark.

Install Steps

  • Add the following in the node bootstrap file for the cluster:
    install.packages("sparklyr")
  • Restart the cluster for this new change to come into effect.

Testing

Type the following in the notebook paragraph and run it: 
%r
library(sparklyr)

If it does not throw any error, the package has been successfully installed on the cluster.

 

 

Have more questions? Submit a request

Comments

  • Avatar
    Pratham Vasa

    It is also possible to install the package like "sparklyr" by adding this package directly from the Notebook interface (like any other packages).

    Open the Notebook and type something like this ->
    ->
    %r
    install.packages(c("tidyr", "sparklyr", "DBI", "dplyr"),repos = "http://cran.us.r-project.org")

Powered by Zendesk