How To: Extracting YARN Logs from Previous Cluster Instances

Description

If you need access to hadoop2 container logs from an earlier instance of a hadoop2 cluster, use the command below. The inputs are:

1. applicationId - get this from within the Job Tracker URL when you click "Tracking URL = Application UI" in the "logs" tab for that job. Should be in the form of "application_1475622964691_0001"
2. cluster id of the hadoop2 cluster running the job, found in the cluster control panel
3. cluster instance id, also found at the end of the Job Tracker URL (clusterInst=273532)
4. user (found at the top of the Job Tracker page after clicking the Job Tracker URL)

The following command can be run directly from a command line on the master node or from the Analyze UI "Shell Command > Bash Command" type:

yarn logs \
  -applicationId application_1475622964691_0001 \
  -logsDir s3://<defloc>/logs/hadoop/<clusterid>/<clusterinstanceid>/app-logs \
  -appOwner <relevant_username>


 
Have more questions? Submit a request

Comments

Powered by Zendesk