Important The KDC along with the database of principals is lost when the master

Important the kdc along with the database of

This preview shows page 236 - 238 out of 395 pages.

Important The KDC, along with the database of principals, is lost when the master node terminates because the master node uses ephemeral storage. If you create users for SSH connections, we recommend that you establish a cross-realm trust with an external KDC configured for high- availability. Alternatively, if you create users for SSH connections using Linux user accounts, automate the account creation process using bootstrap actions and scripts so that it can be repeated when you create a new cluster. Submitting a step to the cluster after you create it or when you create the cluster is the easiest way to add users and KDC principals. Alternatively, you can connect to the master node using an EC2 key pair as the default hadoop user to run the commands. For more information, see Connect to the Master Node Using SSH (p. 313) . The following example submits a bash script configureCluster.sh to a cluster that already exists, referencing its cluster ID. The script is saved to Amazon S3. aws emr add-steps --cluster-id j-01234567 \ --steps Type=CUSTOM_JAR,Name=CustomJAR,ActionOnFailure=CONTINUE,\ Jar=s3:// myregion .elasticmapreduce/libs/script-runner/script-runner.jar,\ Args=[" s3://mybucket/configureCluster.sh "] The following example demonstrates the contents of the configureCluster.sh script. The script also handles creating HDFS user directories and enabling GSSAPI for SSH, which are covered in the following sections. #!/bin/bash 230
Image of page 236
Amazon EMR Management Guide Use Kerberos Authentication #Add a principal to the KDC for the master node, using the master node's returned host name sudo kadmin.local -q "ktadd -k /etc/krb5.keytab host/`hostname -f`" #Declare an associative array of user names and passwords to add declare -A arr arr=([ lijuan ]= pwd1 [ marymajor ]= pwd2 [ richardroe ]= pwd3 ) for i in ${!arr[@]}; do #Assign plain language variables for clarity name=${i} password=${arr[${i}]} # Create a principal for each user in the master node and require a new password on first logon sudo kadmin.local -q "addprinc -pw $password +needchange $name" #Add hdfs directory for each user hdfs dfs -mkdir /user/$name #Change owner of each user's hdfs directory to that user hdfs dfs -chown $name:$name /user/$name done # Enable GSSAPI authentication for SSH and restart SSH service sudo sed -i 's/^.*GSSAPIAuthentication.*$/GSSAPIAuthentication yes/' /etc/ssh/sshd_config sudo sed -i 's/^.*GSSAPICleanupCredentials.*$/GSSAPICleanupCredentials yes/' /etc/ssh/ sshd_config sudo /etc/init.d/sshd restart Adding User HDFS Directories To allow your users to log in to the cluster to run Hadoop jobs, you must add HDFS user directories for their Linux user accounts, and grant each user ownership of their directory. Submitting a step to the cluster after you create it or when you create the cluster is the easiest way to create HDFS directories. Alternatively, you could connect to the master node using an EC2 key pair as the default hadoop user to run the commands. For more information, see Connect to the Master Node Using SSH (p. 313) .
Image of page 237
Image of page 238

You've reached the end of your free preview.

Want to read all 395 pages?

  • Spring '12
  • LauraParker
  • Amazon Web Services, Amazon Elastic Compute Cloud

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Stuck? We have tutors online 24/7 who can help you get unstuck.
A+ icon
Ask Expert Tutors You can ask You can ask ( soon) You can ask (will expire )
Answers in as fast as 15 minutes
A+ icon
Ask Expert Tutors