If you directly delete objects from Amazon S3 that are tracked in EMRFS

If you directly delete objects from amazon s3 that

This preview shows page 70 - 72 out of 395 pages.

If you directly delete objects from Amazon S3 that are tracked in EMRFS metadata, EMRFS treats the object as inconsistent and throws an exception after it has exhausted retries. Use EMRFS to delete objects in Amazon S3 that are tracked using consistent view. Alternatively, you can use the emrfs command line to purge metadata entries for objects that have been directly deleted, or you can sync the consistent view with Amazon S3 immediately after you delete the objects. Topics Enable Consistent View (p. 64) Understanding How EMRFS Consistent View Tracks Objects in Amazon S3 (p. 65) Retry Logic (p. 66) EMRFS Consistent View Metadata (p. 66) Configure Consistency Notifications for CloudWatch and Amazon SQS (p. 68) Configure Consistent View (p. 69) EMRFS CLI Reference (p. 72) Enable Consistent View You can enable Amazon S3 server-side encryption or consistent view for EMRFS using the AWS Management Console, AWS CLI, or the emrfs-site configuration classification. To configure consistent view using the console 1. Open the Amazon EMR console at . 2. Choose Create cluster , Go to advanced options . 3. Choose settings for Step 1: Software and Steps and Step 2: Hardware . 4. For Step 3: General Cluster Settings , under Additional Options , choose EMRFS consistent view . 5. For EMRFS Metadata store , type the name of your metadata store. The default value is EmrFSMetadata . If the EmrFSMetadata table does not exist, it is created for you in DynamoDB. 64
Image of page 70
Amazon EMR Management Guide Consistent View Note Amazon EMR does not automatically remove the EMRFS metadata from DynamoDB when the cluster is terminated. 6. For Number of retries , type an integer value. If an inconsistency is detected, EMRFS tries to call Amazon S3 this number of times. The default value is 5 . 7. For Retry period (in seconds) , type an integer value. This is the amount of time that EMRFS waits between retry attempts. The default value is 10 . Note Subsequent retries use an exponential backoff. To launch a cluster with consistent view enabled using the AWS CLI We recommend that you install the current version of AWS CLI. To download the latest release, see . Note Linux line continuation characters (\) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^). aws emr create-cluster --instance-type m5.xlarge --instance-count 3 --emrfs Consistent= true \ --release-label emr-5.28.1 --ec2-attributes KeyName= myKey To check if consistent view is enabled using the AWS Management Console To check whether consistent view is enabled in the console, navigate to the Cluster List and select your cluster name to view Cluster Details . The "EMRFS consistent view" field has a value of Enabled or Disabled .
Image of page 71
Image of page 72

You've reached the end of your free preview.

Want to read all 395 pages?

  • Spring '12
  • LauraParker
  • Amazon Web Services, Amazon Elastic Compute Cloud

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Stuck? We have tutors online 24/7 who can help you get unstuck.
A+ icon
Ask Expert Tutors You can ask You can ask ( soon) You can ask (will expire )
Answers in as fast as 15 minutes
A+ icon
Ask Expert Tutors