Configuring a Cluster to Auto Terminate or Continue 87 Using Termination

Configuring a cluster to auto terminate or continue

This preview shows page 4 - 7 out of 395 pages.

Configuring a Cluster to Auto-Terminate or Continue ............................................................ 87 Using Termination Protection ............................................................................................. 88 Working with AMIs ................................................................................................................... 92 Using the Default AMI ...................................................................................................... 92 Using a Custom AMI ......................................................................................................... 93 Specifying the Amazon EBS Root Device Volume Size ........................................................... 98 Configure Cluster Software ...................................................................................................... 100 Create Bootstrap Actions to Install Additional Software ....................................................... 100 Configure Cluster Hardware and Networking .............................................................................. 104 Understand Node Types .................................................................................................. 104 Configure EC2 Instances .................................................................................................. 106 Configure Networking ..................................................................................................... 111 Configure Instance Fleets or Instance Groups ..................................................................... 120 Guidelines and Best Practices ........................................................................................... 131 Configure Cluster Logging and Debugging ................................................................................. 136 Default Log Files ............................................................................................................ 136 Archive Log Files to Amazon S3 ....................................................................................... 137 Enable the Debugging Tool .............................................................................................. 138 Debugging Option Information ........................................................................................ 140 Tag Clusters ........................................................................................................................... 140 Tag Restrictions .............................................................................................................. 141 Tag Resources for Billing ................................................................................................. 141 Add Tags to a New Cluster .............................................................................................. 141 Adding Tags to an Existing Cluster .................................................................................... 142 View Tags on a Cluster .................................................................................................... 143 Remove Tags from a Cluster ............................................................................................ 143 Drivers and Third-Party Application Integration .......................................................................... 144 Use Business Intelligence Tools with Amazon EMR .............................................................. 144 Security ......................................................................................................................................... 145 Security Configurations ........................................................................................................... 145 Data Protection ...................................................................................................................... 145 iv
Image of page 4
Amazon EMR Management Guide AWS Identity and Access Management with Amazon EMR ............................................................ 145 Kerberos ................................................................................................................................ 146 Lake Formation ...................................................................................................................... 146 Secure Socket Shell (SSH) ........................................................................................................ 146 Amazon EC2 Security Groups ................................................................................................... 146 Default Amazon Linux AMI Updates .......................................................................................... 146 Use Security Configurations to Set Up Cluster Security ................................................................ 147 Create a Security Configuration ........................................................................................ 147 Specify a Security Configuration for a Cluster .................................................................... 163 Data Protection ...................................................................................................................... 163 Encrypt Data at Rest and in Transit .................................................................................. 164 IAM with Amazon EMR ............................................................................................................ 172 Audience ....................................................................................................................... 172 Authenticating With Identities .......................................................................................... 173 Managing Access Using Policies ........................................................................................ 174 How Amazon EMR Works with IAM ................................................................................... 175 Configure Service Roles for Amazon EMR .......................................................................... 177 Identity-Based Policy Examples ........................................................................................ 203 Authenticate to Cluster Nodes ................................................................................................. 214 Use an Amazon EC2 Key Pair for SSH Credentials ............................................................... 214 Use Kerberos Authentication ............................................................................................ 215 Integrating Amazon EMR with AWS Lake Formation (Beta) ........................................................... 240 Conceptual Overview of Amazon EMR Integration with Lake Formation ................................. 240 Supported Applications and Features ................................................................................ 245 Before You Begin ............................................................................................................ 246 Launch an Amazon EMR Cluster with Lake Formation .......................................................... 252 Control Network Traffic with Security Groups ............................................................................. 258 Working With Amazon EMR-Managed Security Groups ........................................................ 259 Working With Additional Security Groups .......................................................................... 264 Specifying Security Groups .............................................................................................. 264 Security Groups for EMR Notebooks ................................................................................. 266 Using Block Public Access ................................................................................................ 267 Compliance Validation ............................................................................................................. 269 Resilience .............................................................................................................................. 269 Infrastructure Security ............................................................................................................. 269 Connect to Amazon EMR Using an Interface VPC Endpoint .................................................. 270 Manage Clusters ............................................................................................................................. 273 View and Monitor a Cluster ..................................................................................................... 273 View Cluster Status and Details ........................................................................................ 273 Enhanced Step Debugging ............................................................................................... 278 View Application History ................................................................................................. 280 View Log Files ................................................................................................................ 283 View Cluster Instances in Amazon EC2 .............................................................................. 287 CloudWatch Events and Metrics ........................................................................................ 288 View Cluster Application Metrics with Ganglia .................................................................... 310 Logging Amazon EMR API Calls in AWS CloudTrail .............................................................. 311 Connect to the Cluster ............................................................................................................ 313 Connect to the Master Node Using SSH ............................................................................ 313 View Web Interfaces Hosted on Amazon EMR Clusters ......................................................... 318 Terminate a Cluster ................................................................................................................ 326 Terminate a Cluster Using the Console .............................................................................. 326 Terminate a Cluster Using the AWS CLI ............................................................................. 326 Terminate a Cluster Using the API .................................................................................... 327 Scaling Cluster Resources ........................................................................................................ 328 Using Automatic Scaling in Amazon EMR ........................................................................... 328 Manually Resizing a Running Cluster ................................................................................. 336 Cluster Scale-Down ......................................................................................................... 343 v
Image of page 5
Amazon EMR Management Guide
Image of page 6
Image of page 7

You've reached the end of your free preview.

Want to read all 395 pages?

  • Spring '12
  • LauraParker
  • Amazon Web Services, Amazon Elastic Compute Cloud

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Stuck? We have tutors online 24/7 who can help you get unstuck.
A+ icon
Ask Expert Tutors You can ask You can ask ( soon) You can ask (will expire )
Answers in as fast as 15 minutes
A+ icon
Ask Expert Tutors