Error : Could not load history file s3n://<default_location>/logs/hadoop/<cluster_id>/<cluster_inst_id>/mr-history/<job-folder>/<job_id>.jhist

Description:
Sometimes multiple hive queries fail with the above error message where the StackTrace read the following: "Could not load history file s3n://..."

Symptom:
One of the root cause for this issue can be S3 Streaming upload being used for the cluster where the queries were failing.

Due to streaming upload, inputStream gets corrupted and causes this issue.

Action:
Disable S3 Streaming upload by removing the hadoop override "fs.s3n.multipart.uploads.streaming.enabled=true"

from the "Hadoop Override Configuration Variables" field under the cluster settings page for the specific cluster.

Have more questions? Submit a request

Comments

Powered by Zendesk