then this would result in the cluster terminating with the state

Then this would result in the cluster terminating

This preview shows page 384 - 386 out of 395 pages.

, then this would result in the cluster terminating with the state, SHUTDOWN_COMPLETED_WITH_ERRORS . A few ways to troubleshoot this problem include: If you are using an Amazon S3 bucket policy within a VPC, make sure to give access to all buckets by creating a VPC endpoint and selecting Allow all under the Policy option when creating the endpoint. Make sure that any policies associated with S3 resources include the VPC in which you launch the cluster. Try running the following command from your cluster to verify you can access the bucket hadoop fs -copyToLocal s3:// path-to-bucket /tmp/ You can get more specific debugging information by setting the log4j.logger.org.apache.http.wire parameter to DEBUG in /home/hadoop/conf/ log4j.properties file on the cluster. You can check the stderr log file after trying to access the bucket from the cluster. The log file will provide more detailed information: Access denied for getting the prefix for bucket - us-west-2.elasticmapreduce with path samples/wordcount/input/ 15/03/25 23:46:20 DEBUG http.wire: >> "GET /?prefix=samples%2Fwordcount%2Finput %2F&delimiter=%2F&max-keys=1 HTTP/1.1[\r][\n]" 15/03/25 23:46:20 DEBUG http.wire: >> "Host: us- west-2.elasticmapreduce.s3.amazonaws.com[\r][\n]" Errors That Result in START_FAILED Before AMI 3.7.0, for VPCs where a hostname is specified, Amazon EMR maps the internal hostnames of the subnet with custom domain addresses as follows: ip- X.X.X.X.customdomain.com .tld . For example, if the hostname was ip-10.0.0.10 and the VPC has the domain name option set to customdomain.com, the resulting hostname mapped by Amazon EMR would be 378
Image of page 384
Amazon EMR Management Guide AWS GovCloud (US-West) Errors ip-10.0.1.0.customdomain.com . An entry is added in /etc/hosts to resolve the hostname to 10.0.0.10. This behavior is changed with AMI 3.7.0 and now Amazon EMR honors the DHCP configuration of the VPC entirely. Previously, customers could also use a bootstrap action to specify a hostname mapping. If you would like to preserve this behavior, you must provide the DNS and forward resolution setup you require for the custom domain. Cluster Terminated with errors and NameNode Fails to Start When launching an EMR cluster in a VPC which makes use of a custom DNS domain name, your cluster may fail with the following error message in the console: Terminated with errors On the master instance( instance-id ), bootstrap action 1 returned a non-zero return code The failure is a result of the NameNode not being able to start up. This will result in the following error found in the NameNode logs, whose Amazon S3 URI is of the form: s3:// mybucket / logs / cluster- id /daemons/ master instance-id /hadoop-hadoop-namenode- master node hostname .log.gz : 2015-07-23 20:17:06,266 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem (main): Encountered exception loading fsimage java.io.IOException: NameNode is not formatted.
Image of page 385
Image of page 386

You've reached the end of your free preview.

Want to read all 395 pages?

  • Spring '12
  • LauraParker
  • Amazon Web Services, Amazon Elastic Compute Cloud

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Ask Expert Tutors You can ask You can ask ( soon) You can ask (will expire )
Answers in as fast as 15 minutes