This information is available using the DescribeStep and ListSteps API

This information is available using the describestep

This preview shows page 284 - 287 out of 395 pages.

This information is available using the DescribeStep and ListSteps API operations. The FailureDetails field of the StepSummary returned by those operations. To access the FailureDetails information, use the AWS CLI, console, or AWS SDK. To view failure details using the AWS Console 1. Open the Amazon EMR console at . 2. Choose Cluster List and select a cluster. 3. Select the arrow icon next to each step to view more details. If the step has failed and Amazon EMR can identify the root cause, you see the details of the failure. 278
Image of page 284
Amazon EMR Management Guide Enhanced Step Debugging To view failure details using the AWS CLI To get failure details for a step using the AWS CLI, use the describe-step command. aws emr describe-step –cluster-id j-1K48XXXXXHCB –step-id s-3QM0XXXXXM1W The output will look similar to the following: { "Step": { "Status": { "FailureDetails": { "LogFile": "s3://myBucket/logs/j-1K48XXXXXHCB/steps/s-3QM0XXXXXM1W/stderr.gz", "Message": "org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory s3://myBucket/logs/beta already exists", "Reason": "Output directory already exists." }, "Timeline": { "EndDateTime": 1469034209.143, "CreationDateTime": 1469033847.105, "StartDateTime": 1469034202.881 }, "State": "FAILED", "StateChangeReason": {} }, "Config": { "Args": [ "wordcount", "s3://myBucket/input/input.txt", "s3://myBucket/logs/beta" ], "Jar": "s3://myBucket/jars/hadoop-mapreduce-examples-2.7.2-amzn-1.jar", "Properties": {} }, "Id": "s-3QM0XXXXXM1W", "ActionOnFailure": "CONTINUE", "Name": "ExampleJob" 279
Image of page 285
Amazon EMR Management Guide View Application History } } View Application History You can view Spark and YARN application details using the Application history tab of a cluster's detail page in the console. Amazon EMR application history makes it easier for you to troubleshoot and analyze active jobs and job history. The Application history tab provides two viewing options: One-click access to persistent Spark history server – With Amazon EMR version 5.25.0 or later, you can choose the link to access Spark history server UI without setting up a web proxy through an SSH connection. The Spark history server UI provides details about scheduler stages and tasks, RDD sizes and memory usage, environmental information, and information about the running executors. View a summary of the application history – With Amazon EMR version 5.8.0 or later, you can view a summary of the application history in the EMR console including key metrics for stage tasks and executors. The summary of the application history is available for all YARN applications. Additional details are provided for Spark applications, but these details are only a subset of the information available through Spark history server UI. Topics One-click Access to Persistent Spark History Server (p. 280) View a Summary of Application History (p. 282) One-click Access to Persistent Spark History Server With Amazon EMR versions 5.25.0 and later, you can connect to persistent Spark history server from the cluster Summary page or the Application history tab in the console. The one-click access to persistent
Image of page 286
Image of page 287

You've reached the end of your free preview.

Want to read all 395 pages?

  • Spring '12
  • LauraParker
  • Amazon Web Services, Amazon Elastic Compute Cloud

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Stuck? We have tutors online 24/7 who can help you get unstuck.
A+ icon
Ask Expert Tutors You can ask You can ask ( soon) You can ask (will expire )
Answers in as fast as 15 minutes
A+ icon
Ask Expert Tutors