Error: java.lang.OutOfMemoryError: Java heap space

Symptom

While running a query in Hive an error is returned which contains the following phrase:

java.lang.OutOfMemoryError: Java heap space

Cause

This error can occur as a result of a memory failure during the split computation that takes place prior to Mapper tasks loading data from source directories. It is also possible that this occurs due to memory depletion in the Mapper tasks themselves during processing.

Action

If the error occurred prior to the launching of Mapper tasks then the source is likely the split computation and it will be necessary to increase the following two values, restart the cluster and retry the query:

  • HiveClientmemory
  • MaxPermsize

If the error occurred after the launch of Mapper tasks then the source is likely the Mapper memory and it will be necessary to increase the following two values and retry the query:

  • mapreduce.map.memory.mb
  • mapreduce.map.java.opts
Have more questions? Submit a request

Comments

Powered by Zendesk