Cloning a Cluster Using the Console 344 Submit Work to a Cluster 345 Work with

Cloning a cluster using the console 344 submit work

This preview shows page 6 - 8 out of 395 pages.

Cloning a Cluster Using the Console ......................................................................................... 344 Submit Work to a Cluster ........................................................................................................ 345 Work with Steps Using the AWS CLI and Console ................................................................ 345 Submit Hadoop Jobs Interactively .................................................................................... 349 Add More than 256 Steps to a Cluster .............................................................................. 351 Automate Recurring Clusters with AWS Data Pipeline .................................................................. 351 Troubleshoot a Cluster .................................................................................................................... 352 What Tools are Available for Troubleshooting? ........................................................................... 352 Tools to Display Cluster Details ........................................................................................ 352 Tools to View Log Files ................................................................................................... 353 Tools to Monitor Cluster Performance ............................................................................... 353 Viewing and Restarting Amazon EMR and Application Processes (Daemons) ................................... 353 Viewing Running Processes .............................................................................................. 354 Restarting Processes ....................................................................................................... 354 Troubleshoot a Failed Cluster ................................................................................................... 355 Step 1: Gather Data About the Issue ................................................................................. 355 Step 2: Check the Environment ........................................................................................ 356 Step 3: Look at the Last State Change .............................................................................. 357 Step 4: Examine the Log Files .......................................................................................... 357 Step 5: Test the Cluster Step by Step ................................................................................ 358 Troubleshoot a Slow Cluster .................................................................................................... 358 Step 1: Gather Data About the Issue ................................................................................. 359 Step 2: Check the Environment ........................................................................................ 359 Step 3: Examine the Log Files .......................................................................................... 360 Step 4: Check Cluster and Instance Health ......................................................................... 361 Step 5: Check for Arrested Groups .................................................................................... 362 Step 6: Review Configuration Settings ............................................................................... 363 Step 7: Examine Input Data ............................................................................................. 364 Common Errors in Amazon EMR ............................................................................................... 364 Input and Output Errors .................................................................................................. 365 Permissions Errors .......................................................................................................... 367 Resource Errors .............................................................................................................. 367 Streaming Cluster Errors .................................................................................................. 374 Custom JAR Cluster Errors ............................................................................................... 375 Hive Cluster Errors .......................................................................................................... 376 VPC Errors ..................................................................................................................... 377 AWS GovCloud (US-West) Errors ....................................................................................... 379 Other Issues ................................................................................................................... 380 Troubleshoot a Lake Formation Cluster (Beta) ............................................................................ 380 Session Expiration ........................................................................................................... 380 No Permissions for User on Requested Table ...................................................................... 380 Inserting Into, Creating and Altering Tables: Unsupported in Beta ......................................... 381 Write Applications that Launch and Manage Clusters .......................................................................... 382 End-to-End Amazon EMR Java Source Code Sample .................................................................... 382 Common Concepts for API Calls ............................................................................................... 384 Endpoints for Amazon EMR ............................................................................................. 385 Specifying Cluster Parameters in Amazon EMR ................................................................... 385 Availability Zones in Amazon EMR .................................................................................... 385 How to Use Additional Files and Libraries in Amazon EMR Clusters ........................................ 386 Use SDKs to Call Amazon EMR APIs .......................................................................................... 386 Using the AWS SDK for Java to Create an Amazon EMR Cluster ............................................ 386 AWS Glossary ................................................................................................................................. 389 vi
Image of page 6
Amazon EMR Management Guide Overview What Is Amazon EMR? Amazon EMR is a managed cluster platform that simplifies running big data frameworks, such as Apache Hadoop and Apache Spark , on AWS to process and analyze vast amounts of data. By using these frameworks and related open-source projects, such as Apache Hive and Apache Pig, you can process data for analytics purposes and business intelligence workloads. Additionally, you can use Amazon EMR to transform and move large amounts of data into and out of other AWS data stores and databases, such as Amazon Simple Storage Service (Amazon S3) and Amazon DynamoDB. If you are a first-time user of Amazon EMR, we recommend that you begin by reading the following, in addition to this section: Amazon EMR – This service page provides the Amazon EMR highlights, product details, and pricing information. Getting Started: Analyzing Big Data with Amazon EMR (p. 11) – These tutorials get you started using Amazon EMR quickly.
Image of page 7
Image of page 8

You've reached the end of your free preview.

Want to read all 395 pages?

  • Spring '12
  • LauraParker
  • Amazon Web Services, Amazon Elastic Compute Cloud

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Ask Expert Tutors You can ask You can ask ( soon) You can ask (will expire )
Answers in as fast as 15 minutes