To create and work with buckets using the AWS CLI see Using High Level S3

To create and work with buckets using the aws cli see

This preview shows page 48 - 50 out of 395 pages.

To create and work with buckets using the AWS CLI, see Using High-Level S3 Commands with the AWS Command Line Interface in the Amazon Simple Storage Service Console User Guide . To create a bucket using an SDK, see Examples of Creating a Bucket in the Amazon Simple Storage Service Developer Guide . To work with buckets using Curl, see Amazon S3 Authentication Tool for Curl . For more information on specifying Region-specific buckets, see Accessing a Bucket in the Amazon Simple Storage Service Developer Guide . Note If you enable logging for a bucket, it enables only bucket access logs, not Amazon EMR cluster logs. During bucket creation or after, you can set the appropriate permissions to access the bucket depending on your application. Typically, you give yourself (the owner) read and write access and give authenticated users read access. Required Amazon S3 buckets must exist before you can create a cluster. You must upload any required scripts or data referenced in the cluster to Amazon S3. The following table describes example data, scripts, and log file locations. Configure Multipart Upload for Amazon S3 Amazon EMR supports Amazon S3 multipart upload through the AWS SDK for Java. Multipart upload lets you upload a single object as a set of parts. You can upload these object parts independently and in any order. If transmission of any part fails, you can retransmit that part without affecting other parts. After all parts of your object are uploaded, Amazon S3 assembles the parts and creates the object. For more information, see Multipart Upload Overview in the Amazon Simple Storage Service Developer Guide . In addition, Amazon EMR offers properties that allow you to more precisely control the clean up of failed multipart upload parts. The following table describes the Amazon EMR configuration properties for multipart upload. You can configure these using the core-site configuration classification. For more information, see Configure Applications in the Amazon EMR Release Guide . 42
Image of page 48
Amazon EMR Management Guide Prepare Input Data Configuration Parameter Name Default Value Description fs.s3n.multipart.uploads.enabled true A boolean type that indicates whether to enable multipart uploads. When EMRFS Consistent View (p. 63) is enabled, multipart uploads are enabled by default and setting this value to false is ignored. fs.s3n.multipart.uploads.split.size 134217728 Specifies the maximum size of a part, in bytes, before EMRFS starts a new part upload when multipart uploads is enabled. The minimum value is 5242880 (5 MB). If a lesser value is specified, 5242880 is used. The maximum is 5368709120 (5 GB). If a greater value is specified, 5368709120 is used. If EMRFS client-side encryption is disabled and the Amazon S3 Optimized Committer is also disabled, this value also controls the maximum size that a data file can grow until EMRFS uses multipart uploads rather than a PutObject request to upload the file. For more information, see fs.s3n.ssl.enabled true A boolean type that indicates whether to use http or https.
Image of page 49
Image of page 50

You've reached the end of your free preview.

Want to read all 395 pages?

  • Spring '12
  • LauraParker
  • Amazon Web Services, Amazon Elastic Compute Cloud

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Stuck? We have tutors online 24/7 who can help you get unstuck.
A+ icon
Ask Expert Tutors You can ask You can ask ( soon) You can ask (will expire )
Answers in as fast as 15 minutes
A+ icon
Ask Expert Tutors