glm theory - Appendix B Generalized Linear Model Theory We describe the generalized linear model as formulated by Nelder and Wedderburn(1972 and discuss

# glm theory - Appendix B Generalized Linear Model Theory We...

• Notes
• 14

This preview shows page 1 - 4 out of 14 pages.

Appendix B Generalized Linear Model Theory We describe the generalized linear model as formulated by Nelder and Wed- derburn (1972), and discuss estimation of the parameters and tests of hy- potheses. B.1 The Model Let y 1 , . . . , y n denote n independent observations on a response. We treat y i as a realization of a random variable Y i . In the general linear model we assume that Y i has a normal distribution with mean μ i and variance σ 2 Y i N( μ i , σ 2 ) , and we further assume that the expected value μ i is a linear function of p predictors that take values x i = ( x i 1 , . . . , x ip ) for the i -th case, so that μ i = x i β , where β is a vector of unknown parameters. We will generalize this in two steps, dealing with the stochastic and systematic components of the model. G. Rodr´ ıguez. Revised November 2001
2 APPENDIX B. GENERALIZED LINEAR MODEL THEORY B.1.1 The Exponential FamilyWe will assume that the observations come from a distribution in the expo-nential family with probability density functionf(yi) = exp{yiθi-b(θi)ai(φ)+c(yi, φ)}.(B.1)Hereθiandφare parameters andai(φ),b(θi) andc(yi, φ) are known func-tions.In all models considered in these notes the functionai(φ) has theformai(φ) =φ/pi,wherepiis a knownprior weight, usually 1.The parametersθiandφare essentially location and scale parameters.It can be shown that ifYihas a distribution in the exponential family thenit has mean and varianceE(Yi)=μi=b(θi)(B.2)var(Yi)=σ2i=b(θi)ai(φ),(B.3)whereb(θi) andb(θi) are the first and second derivatives ofb(θi). When
B.1. THE MODEL 3 Try to generalize this result to the case where Y i has a normal distribution with mean μ i and variance σ 2 /n i for known constants n i , as would be the case if the Y i represented sample means. Example: In Problem Set 1 you will show that the exponential distribution with density f ( y i ) = λ i exp {- λ i y i } belongs to the exponential family. In Sections B.4 and B.5 we verify that the binomial and Poisson distri- butions also belong to this family. B.1.2 The Link Function
• • • 