How To: Zepplin Parallelism

Description:

Zepplin supports running either Spark SQL or Shell commands in parallel (Scala, Python and R paragraphs are run sequentially).

How To - SQL:

  • Open the Notebook
  • In the top-right select Interpreters.
  • Set zepplin.spark.concurrentSQL to true
  • Set a value for zepplin.spark.sql.maxConcurrency

How To - Shell:

  • Open the Notebook
  • In the top-right select Interpreters.
  • Set zepplin.shell.concurrentCommands to true

Up to 5 Shell Commands may be run concurrently. 

Have more questions? Submit a request

Comments

Powered by Zendesk