ERROR 2043: Unexpected error during Pig execution

Symptom

The follow error is returned during execution:

org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected error during execution.
at org.apache.pig.PigServer.launchPlan(PigServer.java:1284)
at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1258)
at org.apache.pig.PigServer.execute(PigServer.java:1248)

Action

Pig generates some extra counters and due to MAPREDUCE-5875, it can lead to job failures. To work around this on hadoop versions smaller than 2.8 versions, we can follow this : PIG-2208 Here are the key options:

1. Add an integer property (e.g., pig.counter.limit) to the pig property file (e.g., 20). If the number of inputs of a job exceeds this number, the input counters are disabled. Similarly, if the number of outputs of a job exceeds this number, the output counters are disabled.
2. Add a boolean property (e.g., pig.disable.counters) to the pig property file (default: false). If this property is set to true, then the PIG generated counters are disabled.

Have more questions? Submit a request

Comments

Powered by Zendesk