Error: value $ is not a member of StringContext when running a Spark-Scala command

Description:
Executing a Spark-Scala command with a special character fails with the below error.

Example code:

val joinedDF = matchMap.join(qubMap,"custId").sort($"valueinUSD".desc).limit(100000)

error: value $ is not a member of StringContext

Cause:
Spark doesn't recognize the $ sign because the required sql context was not imported.

Resolution:
Reference open source documentation related to sqlContext.implicits.*

https://spark.apache.org/docs/2.0.0/api/java/org/apache/spark/sql/SQLContext.implicits$.html

Have more questions? Submit a request

Comments

Powered by Zendesk