5
Fitting Lecture Distributions to Data (continued)
Announcements
Please see me after class...
If you didn't receive an email from me last week. If you still have problems or questions about registering/switching between/for lecture or lab.
[Introduction]
Graphical Methods
Statistical Tests
Summary
2
Last Time
Gathering data
Determine requirements Identify sources Collect data Make assumptions
Analyzing the data
[Introduction]
Graphical Methods
Statistical Tests
Summary
3
Today
Graphical plotting and comparison
Density/histogram overplot Distribution function differences plot Quantile-Quantile (Q-Q) plot Probability-Probability (P-P) plot
Statistical tests
Chi-square test Kolmogorov-Smirnov test
[Introduction]
Graphical Methods
Statistical Tests
Summary
4
Graphical Methods
Performed by plotting the data and performing an "eyeball" comparison
Density/histogram overplot Distribution function differences plot Quantile-Quantile (Q-Q) plot Probability-Probability (P-P) plot
Implemented fairly easily Microsoft Excel
[Graphical Methods] Statistical Tests Summary
5
Introduction
Density/Histogram Overplot
Plot two series on the same graph
Empirical data as histogram Predicted probability mass in each interval
Get this from the fitted distribution by taking F(right endpoint) F(left endpoint)
Compare both; should be close if the fit is good See first worksheet
[Graphical Methods] Statistical Tests Summary
6
Introduction
Distribution Differences Plot
Idea: plot the difference of two series on a graph
Empirical distribution function
Use cumulative sum of fractions in intervals Use formula for the fitted F
Predicted distribution function
Plot at right endpoints of intervals
See second worksheet
[Graphical Methods] Statistical Tests Summary
7
Introduction
Quantile Quantile Plot
Look at evenly spaced points in [0,1]
For empirical distribution, F-1 of the kth such point should be the kth data point For fitted distribution, we compute F-1 of the points using formula (or approximation) Not against the evenly spaced points
Now plot one of these against the other
If distributions were the same, result should be a straight line
[Graphical Methods] Statistical Tests Summary
8
Introduction
Quantile Quantile Plot (2)
In practice, the distributions are not the same
Result won't be a straight line But it's easy for us to see departures from a straight line
See third worksheet
Note small adjustment (i 0.5)/n to prevent F = 1
[Graphical Methods] Statistical Tests Summary
9
Introduction
Quantile Quantile Plot (3)
In Excel
Create a scatter (X-Y) plot Connect points if you like Add a trend line
This greatly helps in seeing whether the overall slope is correct
If you try to plot these as series, you won't get the right scale
[Graphical Methods] Statistical Tests Summary
10
Introduction
Probability Probability Plot
For this we plot
Sample probability (i 0.5)/n on x-axis Predicted probability F(i-th point) on y-axis
Here also, plot should ideally be a straight line with slope 1, intercept 0 Same basic technique in Excel as for Q-Q plot; trendline is helpful here also See fourth worksheet
[Graphical Methods] Statistical Tests Summary
11
Introduction
Statistical Tests
Formulated as tests of a null hypothesis
H0: Data are samples from IID RV with distribution function equal to the fitted distribution
Two possible results from test:
Reject H0: `Fitted' distribution isn't good Fail to reject H : Not enough evidence to 0 say fitted distribution isn't good
Introduction
Graphical Methods
[Statistical Tests]
Summary
12
Be Careful
There is no outcome `Confirm H0'
The tests never tell you that the data actually come from the fitted distribution They only tell you if it's implausible that the data come from that distribution
These tests rule out some possibilities
Introduction
Graphical Methods
[Statistical Tests]
Summary
13
Chi-square Test
This a is classical test for comparison of empirical data with fitted function
1.
2.
Divide entire range into k intervals, and let Nj be the number of data points in the j-th interval Next, let pj be the proportion of data points that would be in the j-th interval if the fitted distribution were the actual distribution
That is, if H0 were true Note that the expected number of data points in that interval will be npj where n is the total number of points
Graphical Methods [Statistical Tests] Summary
14
Introduction
Chi-square Test (2)
Then we compute
2 k 2
Some things to notice: If the data were a good fit, then we would expect little difference between Nj and n pj Therefore, a large value of 2 means that we have a bad fit We use large values as criteria for rejecting H0
Introduction Graphical Methods [Statistical Tests] Summary
15
= =1 ( N j -np j ) /( np j ) i
Chi-square: Practicalities
Critical region for the test (how large a 2 to use as criterion for rejection) depends on how many parameters we fitted from data
Provided few parameters are fitted, the differences will not be large We generally ignore fitting, using the 2 test with k-1 degrees of freedom
Introduction
Graphical Methods
[Statistical Tests]
Summary
16
Chi-square: Application
We generally set up the intervals to contain equal amounts of probability mass
Easy to do if we can invert the distribution
Number of intervals k should be small enough so that expected number of points npj in each interval is at least 5
Graphical Methods [Statistical Tests] Summary
17
Introduction
Chi-square: Example
See fifth worksheet We used k = 10 intervals, so we expect 10 data points in each (n = 100, pj =.1)
Actual number varies from 5 to 16
Value of 2 is 14.40, not significant at the 90% level
Reject if 2 > 2 (,k-1) (14.40 not > 14.68)
You can get different results with different assumptions
Graphical Methods [Statistical Tests] Summary
18
Introduction
Kolmogorov-Smirnov test
Purpose: check whether it's unlikely that data came from given distribution This test uses the distribution function F instead of the density f
It's exact (if parameters are known), not asymptotic There's no need to group data into intervals
19
K-S test (2)
First suppose the distribution is known, not fitted
This happens: e.g. we could be testing a random-number generator Suppose data are X , ..., ...