DISTRIBUTIONS DESCRIBING NUMERICALLY
1
Numerical Summaries
Describing Data Numerically
Central Tendency Mean Median Mode
Variation Range Interquartile Range Variance Standard Deviation
2
Measures of center: Mode, Median, Mean
Central Tendency
Mean
Median
Mode
x=
!x
i=1
n
i
n
Midpoint of ranked values Most frequently observed value
3
Arithmetic average
Measures of Center: Mode
A measure of central tendency Value that occurs most often Not affected by extreme values Used for either numerical or categorical data There may be no mode There may be several modes
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
0 1 2 3 4 5 6
Mode = 9
No Mode
4
Measures of Center: Median
In an ordered list, the median is the "middle" number (50% above, 50% below)
0 1 2 3 4 5 6 7 8 9 10
0 1 2 3 4 5 6 7 8 9 10
Median = 3
Median = 3
Not affected by extreme values
5
Measures of Center: Median
The Median the point that divides a distribution in half: 50% of the data on the left of the median and the other 50% - on the right. To find the median, arrange the data in ascending (descending) order. If n is odd, the median is the observation with position
n +1 2
If n is even, the median is the average of the middle two values with the positions
n 2
and
n +1 2
6
Model 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Lamborghini Murcielago Ferrari 360 Modena BMW Z8 Mersedes-Benz SL600 Porsche 911 GT2 Mersedes-Benz SL500 Acura NSX BMW M Coupe Ford Thunderbird Chevrolet Corvette Chrysler Prowler BMW Z3 Coupe Porsche Boxter Audi TT Quattro BMW Z3 Roadster Honda S 2000 Mersedes-Benz SLK320 Audi TT Roadster Mazda Miata Mersedes-Benz SLK230 Toyota MR2
City mpg 9 11 13 13 15 16 17 17 17 18 18 19 19 20 20 20 20 22 22 23 25
0% MIN
25% 1st QUARTILE
50% MEDIAN
75% 3rd QUARTILE
100% MAX
7
5 - Number Summary
Max Q3 Median Q1 Min
25 20 18 16 9
5 2 2 7
8
Boxplot is a visual representation of the five number summary.
There are four steps to built a boxplot: 1. Interquartile range IQR= Q3 Q1, shows the spread of the middle 50% of the data 2. Fences
lower fence LF = Q1 1.5(IQR) upper fence UF = Q3 + 1.5(IQR)
3. Whiskers - extend from Q1 and Q3 to the smallest and largest observations within the fences 4. outliers (if any) - extreme observations that fall outside the fences
9
Boxplot: gas mileage for two-seater cars
Collection 1 Box Plot
0
5
10
15
20
25
30
10
City_mpg
5 - Number summary: 20
25 18 9
16
Collection 1 12 10 8 6 4 2
Histogram
Collection 1
Box Plot
8
10
12
14
16
18
20
22
24
26
28
30
0
5
10
15
20
25
30
City_mpg
City_mpg
11
OSCAR's Age
Actresses
22 30 35 40 43 41 26 26 37 26 33 39 35 33 80 25 28 29 29 29 34 31 42 33 63 24 38 27 34 74 29 35 32 38 54 31 27 33 33 35 26 25 24 38 37 50 35 28 31 29 25 29 42 38 45 27 41 46 25 41 61 49 27 30 41 35 36 21 39 28 35 28 60 32 41 34
n=76
12
OSCAR's Age
Actors
44 38 41 49 44 40 51 46 41 34 38 35 62 42 32 40 62 32 42 47 43 36 42 36 52 40 52 31 42 76 54 47 41 43 51 47 48 39 52 29 34 56 35 37 49 53 37 43 34 41 30 57 56 45 38 52 39 39 42 38 36 32 41 49 41 45 60 62 45 37 57 44 42 30 43 60
n=76
13
Academy Award: Ages of Best Actresses and Best Actors
Collection 2 20 18 16 14 12 10 8 6 4 2 0 10 20 30 40 50 60
Histogram
Collection 2
Box Plot
70
80
90
Actresses
0 10 20 30 40 50 60 70 80 90
Actresses
Collection 2 25 20 15 10 5 Histogram
Collection 2
Box Plot
0
10
20
30
40
50
60
70
80
Actors
0 10 20 30 40 50 60 70
14
80
Actors
Measures of Center: The Mean
The arithmetic mean (mean) is the most common measure of central tendency
Observations: x1 x2 x3 ... xn
x1 + x 2 + x 3 + ... + xn 1 n x= = ! " xi n n i=1
15
Measures of Center: The Mean
Affected by extreme values (outliers)
0 1 2 3 4 5 6 7 8 9 10
0 1 2 3 4 5 6 7 8 9 10
Mean = 3
1 + 2 + 3 + 4 + 5 15 = =3 5 5
Mean = 4
1 + 2 + 3 + 4 + 10 20 = =4 5 5
16
Model Lamborghini Murcielago Ferrari 360 Modena BMW Z8 Mersedes-Benz SL600 Porsche 911 GT2 Mersedes-Benz SL500 Acura NSX BMW M Coupe Ford Thunderbird Chevrolet Corvette Chrysler Prowler BMW Z3 Coupe Porsche Boxter Audi TT Quattro BMW Z3 Roadster Honda S 2000 Mersedes-Benz SLK320 Audi TT Roadster Mazda Miata Mersedes-Benz SLK230 Toyota MR2
City mpg 9 11 13 13 15 16 17 17 17 18 18 19 19 20 20 20 20 22 22 23 25
x1 + x 2 + x 3 + ... + xn x= = n 9 + 11 + 13 + ... + 25 374 = = = 17.81 21 21
Mean = 17.81 mpg Compare with the Median = 18 mpg
n =21
17
Center vs. Shape
We can also use the mean and median to help interpret the shape of a distribution In a unimodal distribution:
Left-Skewed
Mean < Median
Symmetric
Mean = Median
Right-Skewed
Median < Mean
18
Measures of Variability
Variation
Range Interquartile Range Variance Standard Deviation
Measures of variation give information on the spread or variability of the data values.
Same center, different variation
19
Measure of Variability : Range
Range = xmax ! xmin
Simplest measure of variation
Example:
0 1 2 3 4 5 6 7 8 9 10 11 12
13 14
Range = 14 - 1 = 13
20
Measure of Variability : The Range
Range = xmax ! xmin
Cars : Range = 25 ! 9 = 16 mpg
21
Disadvantages of the Range
Ignores the way in which data are distributed
7
8
9
10
11
12
7
8
9
10
11
12
Range = 12 - 7 = 5
Range = 12 - 7 = 5
Sensitive to outliers
1,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,3,3,3,3,4,5
Range 5 = - 1 = 4
1,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,3,3,3,3,4,120
Range = 120 - 1 = 119
22
Measure of Variability : Interquartile Range
Can eliminate some outlier problems by using the interquartile range Eliminate high- and low-valued observations and calculate the range of the middle 50% of the data Interquartile range = 3rd quartile 1st quartile IQR = Q3 Q1
23
Interquartile Range
Example: X Q1
25%
minimum
25%
Median (Q2)
25%
Q3
25%
X
maximum
12
30
45
57
70
Interquartile range = 57 30 = 27
24
Measure of Variability : The Variance
Var = s
2
1 2 = ! # ( xi " x ) n " 1 i =1
n
25
Car 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 sum Mean
x
9 11 13 13 15 16 17 17 17 18 18 19 19 20 20 20 20 22 22 23 25 374 17.81
(x ! x)
9 - 17.81 = -8.81 11- 17.81 = -6.81 -4.81 -4.81 -2.81 -1.81 -0.81 -0.81 -0.81 0.19 0.19 1.19 1.19 2.19 2.19 2.19 2.19 4.19 4.19 5.19 7.19
( x ! x )2
77.62 46.38 23.14 23.14 7.90 3.28 0.66 0.66 0.66 0.04 0.04 1.42 1.42 4.80 4.80 4.80 4.80 17.56 17.56 26.94 51.70 319.24
s2 =
( xi ! x )2 " n !1
Cars : x = 17.81 mpg Range = 16 mpg Variance = s 2 = 15.96 mpg 2
Variance
15.96
26
Measure of Variability : The Standard Deviation
s = Variance = s
2
s =
"(x
i
! x)
2
n !1
27
The Standard Deviation: Most commonly used measure of variation Shows variation about the mean Has the same units as the original data
28
Calculation Example: Sample Standard Deviation
Sample Data (xi) : 10 12 n=8 14 15 17 18 18 24
Mean = x = 16
A measure of the "average" scatter around the mean 29
Comparing Standard Deviations
Data A
11 12 13 14 15 16 17 18 19 20 21
Mean = 15.5 s = 3.338 Mean = 15.5
Data B
11 12 13 14 15 16 17 18 19 20 21
s = 0.926
Mean = 15.5 s = 4.570
30
Data C
11 12 13 14 15 16 17 18 19 20 21
Car 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 sum Mean
x
9 11 13 13 15 16 17 17 17 18 18 19 19 20 20 20 20 22 22 23 25 374 17.81
(x ! x)
-8.81 -6.81 -4.81 -4.81 -2.81 -1.81 -0.81 -0.81 -0.81 0.19 0.19 1.19 1.19 2.19 2.19 2.19 2.19 4.19 4.19 5.19 7.19
( x ! x )2
77.62 46.38 23.14 23.14 7.90 3.28 0.66 0.66 0.66 0.04 0.04 1.42 1.42 4.80 4.80 4.80 4.80 17.56 17.56 26.94 51.70 319.24
s = Variance = s 2
Cars : Variance = s 2 = 15.96 mpg 2 St . Deviation = s = 4 mpg
Variance St.Dev
15.96 4.00
31
Measuring variation
Small standard deviation Large standard deviation
32
The Empirical Rule
The empirical rule is useful when talking about a
distribution, using the standard deviation in terms of it's distance from the mean. In general, for symmetric distributions: (x !1s, x + 1s) contains at least 68% of data (x ! 2s, x + 2s) contains at least 95% of data (x ! 3s, x + 3s) contains at least 99.7% of data
NOTE: If the distribution is not unimodal symmetric the empirical rule may not hold.
33
The Empirical Rule
If the data distribution is bell-shaped, then the interval: contains about 68% of the values in x 1s the population or the sample
68%
x 1s
34
The Empirical Rule
x 2s contains about 95% of the values in
the population or the sample contains about 99.7% of the values in the population or the sample
x 3s
95%
99.7%
x 2s
x 3s
35
EXAMPLE: The data being described are the verbal SAT scores for 1025 seniors . Mean = 490 and standard deviation = 100
36
How to estimate the Standard Deviation from the Histogram
The interval
( x ! 2s , x + 2s )
has the length 4 standard deviations and contains at least 95% of data. This interval is approximately equal to the Range of data. Estimation for standard deviation:
Range max ! min s" = 4 4
Note: It is true for symmetrical and unimodal distributions only!
37
Car 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 sum Mean
x 9 11 13 13 15 16 17 17 17 18 18 19 19 20 20 20 20 22 22 23 25 374 17.81
(x-mean) -8.81 -6.81 -4.81 -4.81 -2.81 -1.81 -0.81 -0.81 -0.81 0.19 0.19 1.19 1.19 2.19 2.19 2.19 2.19 4.19 4.19 5.19 7.19
(x-mean)^2 77.62 46.38 23.14 23.14 7.90 3.28 0.66 0.66 0.66 0.04 0.04 1.42 1.42 4.80 4.80 4.80 4.80 17.56 17.56 26.94 51.70 319.24
x = 17.81 mpg Range = 16 Variance = s 2 = 15.96 mpg 2 St . Deviation = s = 4 mpg
Collection 1 12 10 8 6 4 2
Histogram
8
10
12
14
16
18
20
22
24
26
28
30
City_mpg
Variance St.Dev
15.96 4.00
Distribution shape : symmetric Emp. Rule Estimation : Range 25 ! 9 16 s" = = = 4 mpg 4 4 4
38
Common Language and Notations:
Parameters and Statistics
Variables can be summarized using statistics. A statistic is a numerical measure that describes a characteristic of the sample A parameter is a numerical measure that describes a characteristic of the population. We use statistics to estimate parameters
39
A population is an entire group of which we want to characterize. Population parameters: mean, variance, standard deviation, proportion. A sample is a collection of observations on which we measure one or more characteristics. Sample statistics: mean, variance, standard deviation, proportion.
Population
Sample
40
Notations:
Estimation and Inferences
Population
Sample
Population Parameters : Mean Var St. Dev. proportion p
Sample Statistics : Mean Var St. Dev.
2
=
1 # xi N i=1
n
1 n x = ! xi n i=1 1 n s = ! (xi " x)2 n " 1 i=1 s= (x i " x)2 ! n"1
41
1 n 2 ! = # (xi " )2 N i=1
! =
# (xi " )2
N
^ proportion p