of Prediction random variables
Key distinction between fixed and random effects:
Estimate means of fixed effects Estimate variance of random effects
But in some instances, want to predict FUTURE values of a random effect Example (from Efron and Morris, 1975, JASA 70:311-319): Baseball players. Given a player's performance in the beginning of the season, predict performance in rest of season.
c 2011 Dept. Statistics (Iowa State University)
Stat 511 section 22
1 / 15
c 2011 Dept. Statistics (Iowa State University)
Stat 511 section 22
2 / 15
c 2011 Dept. Statistics (Iowa State University)
Stat 511 section 22
3 / 15
c 2011 Dept. Statistics (Iowa State University)
Stat 511 section 22
4 / 15
Best predictor is found by "Shrinking" obs. performance towards overall mean. So, how much shrinkage is needed? How do we compute optimal predictor? General answer using a linear mixed effects model y = X + Zu + , where u N 0 0 , G 0 0 R
Given data y, what is our best guess for values in the unobserved vector u ?
c 2011 Dept. Statistics (Iowa State University)
Stat 511 section 22
5 / 15
Because u is a random vector rather than a fixed parameter, we talk about predicting u rather than estimating u . We seek a Best Linear Unbiased Predictor (BLUP) for u, which we ^ will denote by u To be a BLUP, we require
1. 2. 3. ^ u is a linear function of y. ^ u is unbiased for u so that E(^ - u) = 0. u Var(^ - u) is no "larger" than Var(v - u), where v is any other linear u and unbiased predictor.
It turns out that the BLUP for u is the BLUE of E(u|y).
c 2011 Dept. Statistics (Iowa State University)
Stat 511 section 22
6 / 15
What does E(u|y) look like? We will use the following result about conditional distributions for multivariate normal vectors. Suppose w1 1 11 12 N , w2 2 21 22 where 11 12 21 22 is a positive definite variance matrix.
Then the conditional distribution of w2 given w1 is as follows (w2 |w1 ) N(2 + 21 -1 (w1 - 1 ), 22 - 21 -1 12 ) 11 11
c 2011 Dept. Statistics (Iowa State University)
Stat 511 section 22
7 / 15
Now note that y u Thus, y u N
d
=
X 0
+
Z I I 0
u
X 0 X 0
,
Z I I 0 ,
G 0 0 R
Z I
I 0
= N
ZGZ + R ZG GZ G
c 2011 Dept. Statistics (Iowa State University)
Stat 511 section 22
8 / 15
Thus,
E(u|y) = 0 + GZ (ZGZ + R)-1 (y - X) = GZ -1 (y - X)
Thus, the BLUP of u is ^ GZ -1 (y - X g ) = GZ -1 (y - X(X -1 X)- X -1 y) = GZ -1 (I - X(X -1 X)- X -1 )y For the usual case in which G and = ZGZ + R are unknown, we replace the matrices by estimates and approximate the BLUP of u ^^ ^ ^ -1 by GZ (y - X )
c 2011 Dept. Statistics (Iowa State University)
Stat 511 section 22
9 / 15
Often we wish to make predictions of quantities like C + du for some estimable C. ^ The BLUP of such a quantity is C + d^, the BLUE of C plus d u
g
times the BLUP of u Baseball players example is slightly complicated because quantities of interest are proportional to Binomial random variables. Simpler example, using Normal random variables: An old problem, a variation of a homework problem (# 23 on p. 164) of Mood, A.M. (1950) Introduction to the Theory of Statistics. New York: McGraw-Hill.
c 2011 Dept. Statistics (Iowa State University)
Stat 511 section 22
10 / 15
Suppose intelligence quotients (IQs) for a population students of 2 are normally distributed with a mean and variance u An IQ test was given to an i.i.d. sample of such students. Given the IQ of a student, the test score for that student is normally distributed with a mean equal to the student's IQ and a 2 variance e and is independent of the test score of any other students.
2 2 Suppose it is known that u /e = 9
If the sample mean of the students' test scores was 100, what is the best prediction of the IQ of a student who scored 130 on the test?
c 2011 Dept. Statistics (Iowa State University)
Stat 511 section 22
11 / 15
2 Suppose u1 , . . . , un , N(0, u ) independent of 2 e1 , . . . , en , N(0, e ). i.i.d.
i.i.d.
If we let + ui denote the IQ of student i(i = 1, ..., n), then the IQs 2 of the students are N(, u ) as in the statement of the problem. If we let yi = + ui + ei denote the test score of student 2 i(i = 1, . . . , n), then (yi | + ui ) N( + ui , e ) as in the problem statement. We have y = X + Zu + , where 2 2 2 2 X = 1, = , Z = I, G = u I, R = e I, = ZGZ + R = (u + e ) I. ^ Thus = (X -1 X)- X -1 y = (1 1)- 1 y = . and y GZ
g -1
=
2 u 2 2 u +e
I
2 u 2 2 (y u +e
^ ^ Thus, the BLUP for u is u = GZ -1 (y - X g ) = The ith element of this vector is ^i = u
c 2011 Dept. Statistics (Iowa State University) 2 u 2 + 2 u e
- 1. ) y
(yi - . ). y
Stat 511 section 22 12 / 15
Thus, the BLUP for the IQ of student i, + ui , is 2 2 2 + ^i = . + 2u 2 (yi - . ) = 2u 2 yi + 2e 2 . ^ u y y y + + +
u e u e u e
Note that the BLUP is a convex combination of the individual score and the overall mean score 2 2 u y + 2e 2 . y 2 + 2 i +
u e u e
Because
2 u 2 2 u +e
=
2 u 2 e 2 u 2 +1 e
2 u 2 e
is assumed to be 9, the weights are =
9 9+1
= 0.9 and
2 e 2 2 u +e
= 0.1.
We predict 0.9(130) + 0.1(100) = 127 to be the IQ of a student who scored 130 on the test. US College test results (e.g. SAT) now include information about "If you take this test again, your score is predicted to be ..." If above average, predicted to drop from current score. I suspect these predictions are BLUPs.
c 2011 Dept. Statistics (Iowa State University) Stat 511 section 22 13 / 15
An extension that illustrates an important property of BLUP's IQ problem, except now, yi is the average of ni test scores for a student Some student's scores based on ni = 1 test, others on ni = 5 tests
2 Now, (yi | + ui ) N( + ui , e /ni ) ^ Now, is a weighted average of student scores g
The BLUP for ui is ^i = u
2 u ^ (yi - g ) 2 2 u + e /ni
^ Again a convex combination of yi and overall average, g . But now weights are
2 u 2 + 2 /n u i e
and
2 e /ni 2 + 2 /n u i e
c 2011 Dept. Statistics (Iowa State University)
Stat 511 section 22
14 / 15
Numerical illustration of weights
2 2 2 2 u /e = 9 u /e = 1 ^ ^ Number of tests yi g yi g 1 0.9 0.1 0.5 0.5 2 0.947 0.053 0.667 0.333 3 0.964 0.036 0.75 0.25 4 0.973 0.027 0.80 0.2 10 0.989 0.011 0.909 0.0909 More tests = more precise information about an individual: BLUP is closer to the data value, yi
Fewer tests = less precise information about an individual: ^ BLUP is closer to the estimated population mean, g More variability between individuals: BLUP is closer to the data value, yi
c 2011 Dept. Statistics (Iowa State University) Stat 511 section 22 15 / 15