Virginia Commonwealth University STAT 541 APPLIED STATISTICS FOR ENGINEERS &amp; SCIENTISTS Instructor: Dr. James M. Davenport Lecture # 08 Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 1 Today s Lecture Information in today s lecture corresponds to the following sections in your textbook (in addition to my notes): 6.2 &amp; 6.3: Stem-and-Leaf Plots, Histograms, Numerical Descriptions of Location Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 2 Describing Distributions Graphical Displays -Stem-and-Leaf Plots Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 3 1 Ba be Ruth Ye ar Hom e Runs 1 91 4 1 91 5 1 91 6 1 91 7 1 91 8 1 91 9 1 92 0 1 92 1 1 92 2 1 92 3 0 4 3 2 11 29 54 59 35 41 Ba seball D ata - Hom e Runs per ye ar M icke y Roger M aris L ou G ehrig M antle Ye a r Hom e Runs 1 957 1 958 1 959 1 960 1 961 1 962 1 963 1 964 1 965 1 966 14 28 16 39 61 33 23 26 8 13 Year Hom e Runs 1 923 1 924 1 925 1 926 1 927 1 928 1 929 1 930 1 931 1 932 1 0 20 16 47 27 35 41 46 34 Ye ar Hom e Runs 1 951 1 952 1 953 1 954 1 955 1 956 1 957 1 958 1 959 1 960 13 23 21 27 37 52 34 42 31 40 4 Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 Stem-and-Leaf Plots We must first identify what we call the stems and leaves . The stems are the grouping category and the leaves are elements within the stems. Hence, in this case stems = tens digits leaves = units digits Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 5 Stem-and-Leaf Plot Roger Maris 0 1 2 3 4 5 6 James M. Davenport, Copyright 2008 6 Key: 6|1 means 61 homeruns Lecture 8 - VCU's Stat 541 2 Ordered Stem-and-Leaf Plot for Roger Maris 0 1 2 3 4 5 6 James M. Davenport, Copyright 2008 7 Key: 6|1 means 61 homeruns Lecture 8 - VCU's Stat 541 Back-to-back Stem Plots for Home Runs of Maris and Ruth Key: 6|1 Means 61 Home Runs Ruth Maris 6 4 3 2 0 0 5 8 9 1 1 3 4 6 9 5 2 2 3 6 8 5 4 3 3 9 9 7 6 6 6 1 1 4 9 4 4 5 0 6 1 Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 . 8 Stem-and-Leaf Plots gives a good graphic description of the data, for small to moderately large data sets -- not recommended for extremely large data sets avoids the annoying decisions that lead to non-unique frequency distributions and histograms Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 9 3 Stem-and-Leaf Plots preserves the original data actually gives the order statistics can be considered a grouped graphic display of the order statistics the original order of observation is lost Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 10 Describing Distributions Graphical Displays -Histograms for continuous data Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 11 Frequency Distributions &amp; Continuous Variables Usually we define groups of possible variable values, or classes or intervals or bins when tabulating the frequency distribution of a continuous variable. Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 12 4 Frequency Distributions &amp; Continuous Variables Again, each line in the frequency table represents an interval of values for the variable and the frequencies and relative frequencies are shown. Frequency = count Relative Frequency = fraction [0 to1] Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 13 Frequency Distributions &amp; Continuous Variables The intervals we define MUST: touch, but not overlap extend far enough to include all of the values that we have observed have equal widths (but this can be relaxed) Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 14 Frequency Distributions &amp; Continuous Variables If the intervals are all of equal width, then the height of each bar and the total area of each bar summed together will be proportional to the relative frequency of that interval. Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 15 5 Choosing the Intervals How many intervals? Too few and there is too much compression of the data and information is lost . Too many intervals and there is not enough compression of the data and again we have lost information. Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 16 Choosing the Intervals http://www.stat.sc.edu/~west/javahtml/ click on A Histogram Applet . Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 17 Choosing the Intervals How wide is each Interval? (See the text book, but make it a convenient number.) Where should the interval endpoints fall? What about observations that fall on the interval endpoints (you cannot double count individual data points)? Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 18 6 Choosing Cell Endpoints (three ways) 1. Make intervals closed on the right and open on the left. That is, the right endpoint belongs to that interval, and the left endpoint belongs to the previous interval. i.e. a &lt; X &lt; b Lecture 8 - VCU's Stat 541 or (a,b] 19 James M. Davenport, Copyright 2008 Choosing Cell Endpoints (three ways) 2. Make intervals close on the left and open on the right. That is, the left endpoint belongs to that interval, and the right endpoint belongs to the next interval. I.e. a &lt; X &lt; b Lecture 8 - VCU's Stat 541 or James M. Davenport, Copyright 2008 [a,b) 20 Choosing Cell Endpoints (three ways) 3.Choose the interval endpoints such that it is impossible for any of the data values to fall on the endpoints. For example, if all of the data values are recorded as two digit numbers (e.g. 49), then define the endpoints to be multiples of XX.5 Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 21 7 Choosing Cell Endpoints (three ways) 3. (continued) If the data are recorded to one decimal place, then choose multiples of XXX.X5, etc. etc. However, this does not solve the problem entirely. Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 22 Examples The following examples are constructed using NCSS and the <a href="/keyword/compressive-strength/" >compressive strength</a> of concrete blocks data set (see Data Sets Resources Web Page ). Download it, import it, download the templates, and experiment with it yourself. Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 23 Frequency Distribution Interval 28.0-32.0 32.0-36.0 36.0-40.0 40.0-44.0 44.0-48.0 48.0-52.0 52.0-56.0 56.0-60.0 60.0-64.0 64.0-68.0 Lecture 8 - VCU's Stat 541 Midpoint 30.0 34.0 38.0 42.0 46.0 50.0 54.0 58.0 62.0 66.0 Talley III IIIII III IIIII IIIII IIIII IIIII IIIII I IIIII IIIII IIIII IIIII IIIII IIIII IIIII II IIIII IIII III III I 24 James M. Davenport, Copyright 2008 8 Frequency Distribution fi 3 8 10 16 20 17 9 3 3 1 90 fi/n .0333 .0889 .1111 .1778 .2222 .1889 .1000 .0333 .0333 .0112 1.000 fcumi 3 11 21 37 57 74 83 86 89 90 fcumi/n .0333 .1222 .2333 .4111 .6333 .8222 .9222 .9556 .9889 1.000 25 Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 25.00 Histogram for the <a href="/keyword/compressive-strength/" >compressive strength</a> of Concrete Blocks Percentage 20.00 15.00 10.00 5.00 0.00 26.0 30.0 34.0 38.0 42.0 46.0 50.0 54.0 58.0 62.0 66.0 70.0 <a href="/keyword/compressive-strength/" >compressive strength</a> (from Hogg &amp; Ledolter - page 14-15) (numbers are in 100 pounds per square inch) Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 26 Frequency Polygon A line graph that connects the tops of each of the bars is called a frequency polygon, and can be considered an estimate of the density function which we will discuss later. Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 27 9 25.00 Percentage 20.00 15.00 10.00 5.00 0.00 Histogram for the <a href="/keyword/compressive-strength/" >compressive strength</a> of Concrete Blocks Lecture 8 - VCU's Stat 541 25.00 Percentage 20.00 15.00 10.00 5.00 0.00 Histogram for the <a href="/keyword/compressive-strength/" >compressive strength</a> of Concrete Blocks Lecture 8 - VCU's Stat 541 Lecture 8 - VCU's Stat 541 26.0 26.0 (from Hogg &amp; Ledolter - page 14-15 (numbers are in 100 pounds per square inch) James M. Davenport, Copyright 2008 28 (from Hogg &amp; Ledolter - page 14-15 (numbers are in 100 pounds per square inch) James M. Davenport, Copyright 2008 29 30.0 30.0 34.0 34.0 38.0 38.0 <a href="/keyword/compressive-strength/" >compressive strength</a> <a href="/keyword/compressive-strength/" >compressive strength</a> 42.0 42.0 James M. Davenport, Copyright 2008 46.0 46.0 50.0 50.0 54.0 54.0 58.0 58.0 62.0 62.0 66.0 66.0 70.0 70.0 30 10 Ogive Curve A line graph of the cumulative frequencies (or relative cumulative frequencies) is called an OGIVE CURVE. The points are plotted above the value of the midpoints of each cell. Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 31 100.00 Percentage 80.00 60.00 40.00 20.00 0.00 Histogram for the <a href="/keyword/compressive-strength/" >compressive strength</a> of Concrete Blocks Lecture 8 - VCU's Stat 541 100.00 Percentage 80.00 60.00 40.00 20.00 0.00 Histogram for the <a href="/keyword/compressive-strength/" >compressive strength</a> of Concrete Blocks Lecture 8 - VCU's Stat 541 26.0 26.0 (from Hogg &amp; Ledolter - page 14-15 (numbers are in 100 pounds per square inch) James M. Davenport, Copyright 2008 32 (from Hogg &amp; Ledolter - page 14-15 (numbers are in 100 pounds per square inch) James M. Davenport, Copyright 2008 33 30.0 30.0 34.0 34.0 38.0 38.0 <a href="/keyword/compressive-strength/" >compressive strength</a> <a href="/keyword/compressive-strength/" >compressive strength</a> 42.0 42.0 46.0 46.0 50.0 50.0 54.0 54.0 58.0 58.0 62.0 62.0 66.0 66.0 70.0 70.0 11 What Do These Data Summaries Reveal? Shape: Is the distribution unimodal or bimodal? Is the distribution skewed or symmetric?skewed right? - skewed left? Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 34 What Do These Data Summaries Reveal? Location: Where is the distribution concentrated (i.e. where is its center )? Variation: What assessment can be made as to the spread of the distribution (e.g. range)? Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 35 What Do These Data Summaries Reveal? Unusual features: Where is the distribution smooth and where is it rough ? Are there possible outliers present? Are there gaps in the distribution? Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 36 12 25.00 20.00 Percentage 15.00 10.00 5.00 0.00 Histogram for Average Length of Growing Sea 57 Major U.S. Cities - 1987 Old Farmer's Alma Lecture 8 - VCU's Stat 541 100 Absolute Frequency (Count) 16 14 12 10 8 6 4 2 0 Histogram for Average Number of Days with Thuderstorms Occurring Lecture 8 - VCU's Stat 541 30 Frequency Count 25 20 15 10 5 0 Histogram for Newcomb's Speed of Light Measurements; Data Recorded in 1882 120 140 -40 160 180 Number of Days James M. Davenport, Copyright 2008 37 Number of Days James M. Davenport, Copyright 2008 38 200 220 240 260 280 300 320 340 360 Lecture 8 - VCU's Stat 541 100 95 90 85 80 75 70 65 60 55 50 45 40 35 30 25 20 15 10 5 0 -60 Translated and Scaled Time Measurement Numbers are actually the TIME in seconds, Multiplied by 10 to the 9, minus 24800 James M. Davenport, Copyright 2008 39 -20 0 20 40 60 13 Important Principle If the class intervals are not of equal width, then we must do some additional work. We must change what we plot on the vertical axis so that the area of the bar is proportional to the relative frequency. Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 40 Unequal Interval Widths For most histograms, the total area under the curve will NOT BE EQUAL TO 1. In fact, if the widths of the class intervals are all equal, then the area will be equal to the width of the class intervals. Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 41 Unequal Interval Widths But this is okay since the area of each rectangle is directly proportional to the height of the rectangle, and hence proportional to the relative frequency. (here we have height = relative frequency) Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 42 14 25.00 Histogram for the <a href="/keyword/compressive-strength/" >compressive strength</a> of Concrete Blocks Percentage 20.00 15.00 10.00 5.00 0.00 26.0 30.0 34.0 38.0 42.0 46.0 50.0 54.0 58.0 62.0 66.0 70.0 <a href="/keyword/compressive-strength/" >compressive strength</a> (from Hogg &amp; Ledolter - page 14-15) (numbers are in 100 pounds per square inch) Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 43 Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 44 Unequal Interval Widths For histograms that have unequal width intervals, this is not true. The heights that are plotted must be adjusted in order to preserve the basic principle of area must be proportional to relative frequency . Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 45 15 Unequal Interval Widths What we do is force the total area under the curve to be equal to one. The adjusted height of each rectangle is no longer relative frequency, but will now be defined as DENSITY . It is defined as .. relative frequency density = interval width Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 46 Unequal Interval Widths For a histogram that has density plotted on the vertical axis, the total area of the rectangles of the histogram will be equal to 1 (I.e. the total area under the curve is equal to 1). (here we have area = relative frequency) Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 47 Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 48 16 Histogram of CO Emissions: Equal Bin Widths 10.00 8.00 Density 6.00 4.00 2.00 0.00 -5 0 5 10 15 20 25 30 CO Emissions in Grams per Mile Driven Note: Density is the same as Frequency Since the Bin Width Is Equal to 1 Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 49 Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 50 Describing Distributions Numerical Summaries Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 51 17 Numerical Descriptions Frequently, we also want to summarize the information in a distribution with numerical, descriptive measures, that (hopefully) will help us in our quest to describe the distribution . Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 52 Numerical Descriptions Measures of Location Measures of Variation Measures of Relative Standing Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 53 Measure of Location Arithmetic Mean Median Mode Mid-range Trimmed Mean Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 54 18 Notation Repeated observed values of a measurement variable are denoted as: X1, X2, X3, X4, . . . , Xn The actual observations are denoted: x1, x2, x3, x4, . . . , xn Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 55 Mean - definition The mean (or arithmetic average) of a set of observations denoted by X1, X2, X3, X4, . . . , Xn is given by X= Lecture 8 - VCU's Stat 541 X i =1 n i n = X1 + + X n n 56 James M. Davenport, Copyright 2008 Mean We interpret the mean as the center of mass or the balance point of the distribution. Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 57 19 Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 58 Mean Note: the sample mean is sensitive to the influence of extreme observations. Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 59 Mean Company ZYXWVU has five employees; the owner/boss &amp; 4 workers. Each worker has a salary of $20,000/year Boss has a salary of $920,000/year Average salary for the company is $200,000 (not very realistic) Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 60 20 A Resistant Measure We need a measure of the center of a distribution that is more resistant: one that limits the influence of extreme observations one that does not undergo a large change whenever there are changes in a few of the observations Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 61 Median The MEDIAN is such a resistant measure of the center or location of a distribution (at least more so than the mean). Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 62 Median: Definition The median is the middle of the distribution (i.e. that number which divides the distribution into two equal parts. Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 63 21 Median: Definition If n is an odd number, then take the middle observation as the sample median. If n is an even number, then take the average of the two middle observations as the sample median. Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 64 Median: Definition It is obvious by the second item in the previous slide that the median is not a unique number. In fact, there may be an infinite number of medians for such data distributions. Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 65 Example: Ruth &amp; Mantle (the Yankee Years) Ruth n = 15 22 25 34 35 41 41 46 46 46 47 49 54 54 59 60 Mantle n = 18 13 15 18 19 21 22 23 23 27 30 31 34 35 37 40 42 52 54 Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 66 22 Calculation of the Median Using Order Statistics Example: <a href="/keyword/compressive-strength/" >compressive strength</a> of Concrete Blocks n = 90 n n 90 + 1 = 46 = = 45 2 2 2 45.3 + 45.7 M= = 45.5 2 Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 M= X (45) + X (46) 2 X = 45.5067 67 Calculation of the Median Using Order Statistics If n is an odd integer, the median is M = X n +1 2 For example, if n = 15, then So, M = X (8) . Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 n + 1 15 + 1 = =8 . 2 2 68 Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 69 23 What to do with Outliers. Strategies for dealing with extreme observations -- First, detect the outlier and investigate its cause. If the observation is in error, then correct it. Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 70 What to do with Outliers. If there is an extremely good reason, then the observation may be deleted, but you almost NEVER want to do this. Leave the outlier as is and give both the usual measure of location (sample mean) along with a more resistant measure of location (such as the median). Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 71 Other Measures of Location -- Mode The MODE is the most frequently occurring observation or the class interval (we usually take the mid-point of this interval) with the highest relative frequency. Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 72 24 Other Measures of Location -- Midrange The MIDRANGE is defined to be the average of the largest and smallest order statistics. Midrange = X (1) + X ( n ) 2 73 Lecture 8 - VCU's Stat 541 James M. Davenport, Copyright 2008 Other Measures of Location - Trimmed Mean For example, a 10% trimmed mean is computed as follows: Suppose n = 100 X 10% trimmed = Lecture 8 - VCU's Stat 541 X ( 11) + + X ( 90 ) 80 James M. Davenport, Copyright 2008 74 25 Baseball Data - Home Runs per year Babe Ruth Year Home Runs 0 4 3 2 11 29 54 59 35 41 46 25 47 60 54 46 49 46 41 34 22 6 Roger Maris Year Runs 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 14 28 16 39 61 33 23 26 8 13 9 5 Home Lou Gehrig Year Runs 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 Home Runs 1 0 20 16 47 27 35 41 46 34 32 49 30 49 37 29 0 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 13 23 21 27 37 52 34 42 31 40 54 30 15 35 19 23 22 18 Mickey Mantle Year Home 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 Source: Macmillan Baseball Encyclopedia, 9th Edition. washingtonpost.com Law of Averages (Editorial) (comments and bolding added by nwh) Friday, February 21, 2003; Page A26 AT A HIGH SCHOOL in Kennesaw, Ga., yesterday to sell his tax cuts, President Bush repeated several of his favorite sound-bite statistics to argue that his plan would help ordinary Americans, small-business owners and senior citizens. But as any math teacher there could have attested (well, any statistics teacher), Mr. Bush's arguments rely on a misleading use of averages to make his foolhardy plan appear fair. &quot;Under this plan, 92 million Americans receive an average tax cut of $1,083,&quot; Mr. Bush said. &quot;That's fair.&quot; No, it's deceptive. The vast majority of taxpayers -- 80 percent -- would receive less than that amount, according to data from the Urban Institute-Brookings Institution Tax Policy Center. For the truly typical household -- filers in the middle fifth of the income spectrum -- the average tax cut would be $256. Almost half of all taxpayers would see their taxes drop by less than $100. At the top of the income pyramid, however, the tax savings would be huge; the top 1 percent of filers would receive an average tax cut of $24,100. The average tax cut touted by Mr. Bush is more than $1,000 only because the savings for the wealthiest Americans are so large. &quot;We estimate that 23 million small-business owners across America will receive an average income tax rate cut of $2,042,&quot; Mr. Bush said. &quot;That matters.&quot; Again, misleading. As with the individual taxpayer statistics, the Tax Policy Center estimates that nearly four out of five tax filers with small-business income would receive less than that amount. More than half would receive $500 or less. Nearly a quarter would receive no tax cut at all -- a group that doesn't drag down Mr. Bush's average because it's simply not included in the calculation. But a small number of wealthy individuals with small-business income would receive huge tax cuts, once again inflating the average. &quot;It means that 10 million seniors, nearly one in four, who receive dividend income will get relief,&quot; Mr. Bush said of his plan to cut dividend taxes. &quot;Now, that's important. . . . Getting rid of the double taxation of dividends is an incredibly positive thing for the quality of life of our seniors.&quot; Some seniors would see their quality of life improve a lot more than others, however. You can probably guess which ones. A big slice of the dividend tax cut -- 37 percent -- would indeed go to seniors. But the majority of elderly people -- the two-thirds with incomes below $50,000 -- would save on average $325 or less. Meanwhile, a small number of high-income elderly would reap most of the benefits. More than three-quarters of the part of the dividend tax break that would go to the elderly would flow to the 19 percent of senior citizens with incomes above $75,000; 43 percent of the benefits would go to the richest in that group, the 2.5 percent of senior citizens with incomes greater than $200,000. They would save an average of more than $5,000. Mr. Bush must know how phony his &quot;averages&quot; are. Any time a salesman has to resort to such deceptive tactics, the customer ought to be wary about what is being sold. 2003 The Washington Post Company More on Averages (Means)