This preview shows page 287 - 290 out of 395 pages.
•If you reconfigure Spark applications in a running cluster, the application history will be not availablethrough the Spark history server UI.•For each AWS account, the number of active Spark history server UIs cannot exceed 50.•You can access Spark history server UI from the console in the US East (N.Virgina and Ohio), US West(N.California and Oregon), Canada (Central), EU (Frankfurt, Ireland, and London), and Asia Pacific(Mumbai, Seoul, Singapore, Sydney, and Tokyo) Regions.Access Application History through Spark History Server UIOn the Application historytab or the cluster Summarypage for your cluster in the Amazon EMRconsole, choose the Spark history server UIlink.The Spark history server UI opens in a new browser tab. This web interface displays the sameinformation as the open source Spark HistoryServer UI if you set up a web proxy through an SSHconnection. For more information, see Monitoring and Instrumentation.You can view YARN container logs through the links on the Spark history server UI.NoteTo access YARN container logs from the Spark history server UI, you must enable logging toAmazon S3 for your cluster. If logging is not enabled, the links to YARN container logs will notwork.281
Amazon EMR Management GuideView Application HistoryView a Summary of Application HistoryWith Amazon EMR versions 5.8.0 and later, you can view a summary of the application history from theApplication historytab in the Amazon EMR console. Amazon EMR keeps the summary of applicationhistory for seven days after an application has completed.The following sequence demonstrates a drill-down through a Spark or YARN application into job detailsusing the Application historyon a cluster details page. To view cluster details, from the Clusterslist,select a cluster Name. To view information about YARN container logs, you must enable logging foryour cluster. For more information, see Configure Cluster Logging and Debugging. For Spark applicationhistory, the information provided in the summary table is only a subset of the information availablethrough Spark history server UI.In the following Application historytab example, two rows are expanded to show the diagnosticsummaries for two different Spark applications, and an Application IDis selected to view furtherapplication detail.On the Jobstab of YARN applicationdetails, the description of Job 0 is selected to see Job 0 details.On the Job 0 details page, information about individual job stages is expanded, and then the Descriptionfor Stage 1 is selected to see Stage 1 details.282
Amazon EMR Management GuideView Log FilesOn the Stage 1details page, key metrics for stage tasks and executors can be seen, and task andexecutor logs can be viewed using links.
As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.
Temple University Fox School of Business ‘17, Course Hero Intern
I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.
University of Pennsylvania ‘17, Course Hero Intern
The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.
Tulane University ‘16, Course Hero Intern
Ask Expert Tutors
You can ask
You can ask ( soon)
You can ask
(will expire )