NodeManager checks the health of the disks determined by

Nodemanager checks the health of the disks determined

This preview shows page 378 - 380 out of 395 pages.

NodeManager checks the health of the disks determined by yarn.nodemanager.local-dirs and yarn.nodemanager.log-dirs . The checks include permissions and free disk space (< 90%). If a disk fails the check, the NodeManager stops using that particular disk but still reports the node status as healthy. If a number of disks fail the check, the node is reported as unhealthy to the ResourceManager and new containers are not assigned to the node. The application master can also blacklist a NodeManager node if it has more than three failed tasks. You can change this to a higher value using the mapreduce.job.maxtaskfailures.per.tracker configuration parameter. Other configuration settings you might change control how many times to attempt a task before marking it as failed: for map tasks and mapreduce.reduce.maxattempts for reduce tasks. For more information about changing configuration settings, see Configuring Applications in the Amazon EMR Release Guide . Throttling Errors The errors "Throttled from Amazon EC2 while launching cluster" and "Failed to provision instances due to throttling from Amazon EC2 " occur when Amazon EMR can not complete a request because another service has throttled the activity. Amazon EC2 is the most common source of throttling errors, but other services may be the cause of throttling errors. AWS service limits apply on a per-Region basis to improve performance, and a throttling error indicates that you have exceeded the service limit for your account in that Region. 372
Image of page 378
Amazon EMR Management Guide Resource Errors Possible Causes The most common source of Amazon EC2 throttling errors is a large number of cluster instances launching so that your service limit for EC2 instances is exceeded. Cluster instances may launch for the following reasons: New clusters are created. Clusters are resized manually. For more information, see Manually Resizing a Running Cluster (p. 336) . Instance groups in a cluster add instances (scale out) as a result of an automatic scaling rule. For more information, see Understanding Automatic Scaling Rules (p. 329) . Instance fleets in a cluster add instances to meet an increased target capacity. For more information, see Configure Instance Fleets (p. 121) . It is also possible that the frequency or type of API request being made to Amazon EC2 causes throttling errors. For more information about how Amazon EC2 throttles API requests, see Query API Request Rate in the Amazon EC2 API Reference . Solutions Consider the following solutions: Create a support case to request a Service limit increase . If you have clusters that launch on the same schedule—for example, at the top of the hour—consider staggering start times. If you have clusters that are sized for peak demand, and you periodically have instance capacity, consider specifying automatic scaling to add and remove instances on-demand. In this way, instances are used more efficiently, and depending on the demand profile, fewer instances may be requested at a given time across an account. For more information, see
Image of page 379
Image of page 380

You've reached the end of your free preview.

Want to read all 395 pages?

  • Spring '12
  • LauraParker
  • Amazon Web Services, Amazon Elastic Compute Cloud

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Ask Expert Tutors You can ask You can ask ( soon) You can ask (will expire )
Answers in as fast as 15 minutes