This preview shows page 358 - 360 out of 395 pages.
Topics•What Tools are Available for Troubleshooting? (p. 352)•Viewing and Restarting Amazon EMR and Application Processes (Daemons) (p. 353)•Troubleshoot a Failed Cluster (p. 355)•Troubleshoot a Slow Cluster (p. 358)•Common Errors in Amazon EMR (p. 364)•Troubleshoot a Lake Formation Cluster (Beta) (p. 380)When you are developing a new Hadoop application, we recommend that you enable debugging andprocess a small but representative subset of your data to test the application. You may also want to runthe application step-by-step to test each step separately. For more information, see Configure ClusterLogging and Debugging (p. 136) and Step 5: Test the Cluster Step by Step (p. 358).What Tools are Available for Troubleshooting?There are several tools you can use to gather information about your cluster to help determine whatwent wrong. Some require that you initialize them when you launch the cluster; others are available forevery cluster.Topics•Tools to Display Cluster Details (p. 352)•Tools to View Log Files (p. 353)•Tools to Monitor Cluster Performance (p. 353)Tools to Display Cluster DetailsYou can use the AWS Management Console, AWS CLI, or EMR API to retrieve detailed information aboutan EMR cluster and job execution. For more information about using the AWS Management Console andAWS CLI, see View Cluster Status and Details (p. 273).Amazon EMR Console Details PaneIn the Clusterslist on the Amazon EMR console you can see high-level information about the status ofeach cluster in your account and region. The list displays all clusters that you have launched in the pasttwo months, regardless of whether they are active or terminated. From the Clusterslist, you can select acluster Nameto view cluster details. This information is organized in different categories to make it easyto navigate.The Application historyavailable in the cluster details page can be particularly useful fortroubleshooting. It provides status of YARN applications, and for some, such as Spark applications youcan drill into different metrics and facets, such as jobs, stages, and executors. For more information, seeView Application History (p. 280). This feature is available only in Amazon EMR version 5.8.0 and later.352
Amazon EMR Management GuideTools to View Log FilesAmazon EMR Command Line InterfaceYou can locate details about a cluster from the CLI using the --describeargument.Amazon EMR APIYou can locate details about a cluster from the API using the DescribeJobFlowsaction.Tools to View Log FilesAmazon EMR and Hadoop both generate log files as the cluster runs. You can access these log files fromseveral different tools, depending on the configuration you specified when you launched the cluster. Formore information, see Configure Cluster Logging and Debugging (p. 136).
As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.
Temple University Fox School of Business ‘17, Course Hero Intern
I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.
University of Pennsylvania ‘17, Course Hero Intern
The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.
Tulane University ‘16, Course Hero Intern
Stuck? We have tutors online 24/7 who can help you get unstuck.
Ask Expert Tutors
You can ask
You can ask
You can ask
(will expire )