3.1.5.5 Lab - Correlation Analysis in Python.pdf - 3.1.5.5...

3.1.5.5 Lab - Correlation Analysis in Python
October 11, 2021
1 Lab - Correlation Analysis in Python
1.0.1 Objectives
* Part 1: The Dataset * Part 2: Scatterplot Graphs and Correlatable Variables * Part 3: Calculating
Correlation with Python * Part 4: Visualizing
1.0.2 Scenario/Background
Correlation is an important statistical relationship that can indicate whether the variable values
are linearly related.
In this lab, you will learn how to use Python to calculate correlation. In Part 1, you will setup the
dataset. In Part 2, you will learn how to identify if the variables in a given dataset are correlatable.
Finally, in Part 3, you will use Python to calculate the correlation between two sets of variable.
1.0.3 Required Resources
• 1 PC with Internet access
• Raspberry Pi version 2 or higher
• Python libraries: pandas, numpy, matplotlib, seaborn
• Datafiles: brainsize.txt
1.1 Part 1: The Dataset
You will use a dataset that contains a sample of 40 right-handed Anglo Introductory Psychology
students at a large Southwestern university. Subjects took four subtests (Vocabulary, Similarities,
Block Design, and Picture Completion) of the Wechsler (1981) Adult Intelligence Scale-Revised.
The researchers used Magnetic Resonance Imaging (MRI) to determine the brain size of the subjects.
Information about gender and body size (height and weight) are also included. The researchers
1

withheld the weights of two subjects and the height of one subject for reasons of confidentiality.
Two simple modifications were applied to the dataset:
1. Replace the quesion marks used to represent the withheld data points described above by the
‘NaN’ string. The substitution was done because Pandas does not handle the question marks
correctly.
2. Replace all tab characters with commas, converting the dataset into a CSV dataset.
The prepared dataset is saved as brainsize.txt .
Step 1: Loading the Dataset From a File. Before the dataset can be used, it must be loaded
onto memory.
In the code below, The first line imports the pandas modules and defines pd as a descriptor that
refers to the module.
The second line loads the dataset CSV file into a variable called brainFile .
The third line uses read_csv() , a pandas method, to convert the CSV dataset stored in brainFile
into a dataframe. The dataframe is then stored in the brainFrame variable.
Run the cell below to execute the described functions.
[3]: # Code cell 1
import pandas as pd
brainFile = './Data/brainsize.txt'
brainFrame = pd . read_csv(brainFile)
Step 2: Verifying the dataframe. To make sure the dataframe has been correctly loaded and
created, use the head() method. Another Pandas method, head() displays the first five entries of
a dataframe.