How to: Create a Temporaray table in Spark SQL

This article provides an example to create a temporary table for Spark SQL. 


-> import scala.math.random

//create a temp table records
-> case class Record(age: Int, married: Int)
-> val rdd = sc.parallelize((1 to 10000).map(i => Record((random*100).toInt, (random*2).toInt)))

//Assuming we are using hive on spark sql
-> val q = hiveContext.applySchema(rdd,rdd.schema)

// Any RDD containing case classes can be registered as a table. The schema of the table is automatically inferred using scala reflection.
-> q.registerTempTable("records")


-> select age,count(*) from records where age <=${maxAge=30}group by age order by age

Have more questions? Submit a request


Powered by Zendesk