L8handout - PUBH 7430 Lecture 8 J. Wolfson Division of...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: PUBH 7430 Lecture 8 J. Wolfson Division of Biostatistics University of Minnesota School of Public Health September 29, 2011 Beyond the linear model Recall the two key assumptions of the linear model: 1 2 Mean is a linear combination of predictors Variance is a constant not depending on mean Limitation Not all types of outcomes satisfy these assumptions Beyond the linear model Example 1: Binary data If Yi can assume only two possible values (usually 0/1), then it is usual to assume that Yi follows a Bernoulli distribution with success probability pi . E (Yi ) = P (Yi = 1) = pi Var (Yi ) = pi (1 − pi ) Beyond the linear model Suppose we set up a linear model for Y: E (Yi | xi ) = xi β Var (Yi | xi ) = σ 2 Beyond the linear model TRUTH MODEL Yi ∼ Bern(pi ) – pi = xi β E (Yi | xi ) = pi Var (Yi | xi ) = pi (1 − pi ) Var (Yi | xi ) = σ 2 Beyond the linear model TRUTH MODEL Yi ∼ Bern(pi ) – pi = xi β E (Yi | xi ) = pi Var (Yi | xi ) = pi (1 − pi ) Var (Yi | xi ) = σ 2 Problem 1: mean model • Linear predictor xi β can take on values between −∞ and +∞ • But E (Yi | xi ) = P (Yi = 1 | xi ) = pi is a probability, which can only take values between 0 and 1 • Inconvenient, but not disastrous Beyond the linear model TRUTH MODEL Yi ∼ Bern(pi ) – pi = xi β E (Yi | xi ) = pi Var (Yi | xi ) = pi (1 − pi ) Var (Yi | xi ) = σ 2 Problem 2: variance assumption • Assumed that variance was constant, whereas it is actually a function of the mean: Var (Yi | xi ) = pi (1 − pi ) = E (Yi | xi )[1 − E (Yi | xi )] • Potentially disastrous: Estimated variances (hence ˆ confidence intervals, p-values) of β can be way off, can get bias in some cases Beyond the linear model Example 2: Count data • Common in statistics/biostatistics (eg. number of visits to a doctor’s visits, number of relapses, etc.) • Underlying process is often assumed to be Poisson (distribution on the integers 0, 1, 2, . . . ) For Yi ∼ Poisson(rate = λi ), E (Yi ) = λi Var (Yi ) = λi Beyond the linear model TRUTH Yi ∼ Poisson(λi ) E (Yi | xi ) = λi Var (Yi | xi ) = λi MODEL – λi = xi β Var (Yi | xi ) = σ 2 As before, fitting a linear model to Poisson data is problematic: 1 2 xi β ∈ (−∞, +∞), but λi (a rate) must be positive Constant variance assumption is clearly violated since Var (Yi | xi ) = E (Yi | xi ) = λi Violating variance assumptions Making incorrect assumptions about variance can lead to: • Wrong confidence intervals • Incorrect p-values • Biased effect estimates There are two common ways in which variance assumptions are wrong: 1 2 We assume NO mean-variance relationship when one actually exists We assume the WRONG FORM for the mean-variance relationship Mean-variance relationships • Consider a random variable Y with a N (µ, σ 2 ) distribution • Clearly, E (Y ) tells us nothing about Var (Y ), and hence there is no relationship between the mean and variance • But if X has (say) a Poisson distribution with mean λ, then E (X ) = Var (X ) = λ and there is a mean-variance relationship • Mean-variance relationship may be more complex, eg. for W ∼ Gamma(α, β ), E (W ) = αβ and Var (W ) = αβ 2 = β E (W ) Mean-variance relationships 30 Simulated Poisson data q qq 25 qq qq q q qq q 20 q qq qq q q 15 q 10 qq q qq q q q q qq q qqqq q q q qq qq qq q qq q q q qq q q qq qq q q q q q q q q q q q qqqq q qq qq q q q q q q q q qq q q q q qq qq q q qq q q qq q q qq q qqq qqq qq q q q qq q q q q qq q q q q q qq q qq q qq q q q q q qq q q qq q q q q qq qq q q qq q qq q q qq q q q qq q q qq q qq q q q q q q q qq qq q q q q q qq q q qq q q q q qq q q q q q q q q qq q qq q q q q q q q q qq q q qq q q qq q qqq qqq q qq q q qq qq q q q q q qq q qq q qq q q q q q qq q qqq qqq qq q q qqq qq q q qqq qq q qqq q q qq q q q q q q q q qq q q q qq q q q qq q q q q q qq q q qq qq q q qq q qq q q q q qq qqqq q qq q q q qq q q qq q q q q q q qq q qq q q q q q q qq q q q qq q qqq q q q qq q qq q q q qq q q q q q qq q q q q q qq qq qq qq q q qq 5 Y qq q q q q q q q q qq q q qq qq q q q qq qq q qq q qq q q q qq q qq q q q q q q qq 0 q µ + −2 µ q q q q q qqq qq q qq q q 5 10 15 µ 20 Mean-variance relationships 30 Simulated Poisson data q qq 25 qq qq q µ + −2 µ µ + − 2 10 q qq q 20 q q qq qq q q 15 q 10 qq q qq q q q q qq q qqqq q q q qq qq qq q qq q q q qq q q qq qq q q q q q q q q q q q qqqq q qq qq q q q q q q q q qq q q q q qq qq q q qq q q qq q q qq q qqq qqq qq q q q qq q q q q qq q q q q q qq q qq q qq q q q q q qq q q qq q q q q qq qq q q qq q qq q q qq q q q qq q q qq q qq q q q q q q q qq qq q q q q q qq q q qq q q q q qq q q q q q q q q qq q qq q q q q q q q q qq q q qq q q qq q qqq qqq q qq q q qq qq q q q q q qq q qq q qq q q q q q qq q qqq qqq qq q q qqq qq q q qqq qq q qqq q q qq q q q q q q q q qq q q q qq q q q qq q q q q q qq q q qq qq q q qq q qq q q q q qq qqqq q qq q q q qq q q qq q q q q q q qq q qq q q q q q q qq q q q qq q qqq q q q qq q qq q q q qq q q q q q qq q q q q q qq qq qq qq q q qq 5 Y qq q q q q q q q q qq q q qq qq q q q qq qq q qq q qq q q q q qq q qq q q 0 q q q qq q q q q q qqq qq q qq q q 5 10 15 µ 20 Overdispersion Sometimes, count or other data may be overdispersed with respect to the assumed variance, i.e. VarT (Yi ) > VarA (Yi ) (VarT = true variance, VarA = assumed variance) Example We assume count data are Poisson (so VarA (µi ) = µi ), but in fact VarT (µi ) = 3µi Note: Underdispersion (VarT < VarA ) does happen, but less frequently Overdispersion q Simulated Overdispersed Poisson data q 30 q q q q qq q q q 25 q 20 Y 15 10 5 q q q q qq q qq q q qq q q q q q q qq q q q qq qq q q qq q qq q q q qq q q qq q qqq q q q q qq qq q q q qq q q q q q q q q q q q q q q qq q q q qq qq q qq q qq qq q qq q q q qq q q q qq q q q q qq q qq qq qq qq q q qq qq q qq q q q q q q q qqq q q q qq q q q q qq qq qq qq q qq q q qq q q q qq q q q q qq q qqq q q qq q q q qq q q q q q qq q q q q qq q q qq q qq q qq q q q qq qq q qq q q q q qq q q q q q q q qq q q q q q qq q qq qq qq q q q q q q q q qq q q q q q q q qq q q qqq q q q qq qq q qq q q q q q qqq q q q q q qq qq q qq q q q q q q q q q q qq q q q q q q q q qq q qq q qq q q qqq q q q q q q qq q qq qq q q q q q qq q qqq q qq q qq qq q q qq q q q qq qq q q q qq qqq qq q q q qq qq q qq q q qq q qq qq qq qq q qq q q q q q q qqq qq q q qqq qq qq q q q q q q qq q qq q q q qq q qq qq qq q q q q q qq q q q qq qqqq q q q q q q q q qqqq q qq qq q qq q q q q q q q qq q q q qq q q q qq q q q q q q q q q q qq qq q q qq qq qqqq q qq q qq q q qq qq qq qq q q qq q q q q q q q q qq q q qq q q q qq q q q q q q q q qq q qqq q q q q q q qq q qq q q q q qq qq qq q qq q qqq q qq qq q q q q qq q qq q qq q q q q q qq q q q q qq q q q qqqq q qq qq qq q q q q qq q qq q q q q q q qq q qq q q q q qq q qq q q q q q qq q q q qq q qq qqq q q q q q qq q q q q q qq q q q q q q q q q qq q q q q q qqq qq qq q qqqq q q qq q q q qq qq q qq q qqq q q qq q q qq qq qq qq q q q q q q q q q q qqq q q q q q qq q q q qqq q q q q q q qqqqqq qq q qqqqqq q q q q q qq q q q q qq q qq qqq q q q qq q qq q q qq q q q q q qqq qq q q q q q qq qqq qq q q q qq q q qq q q q q q q q q q q qq q q q q q q q qq q q q q q qq q qq q qq q q qq q qq q qq qq qq q q q q q qq q q q q q q q q q q q q q 0 q q qq q q q q q q q µ + −2 µ q q q 5 q q q q 10 q qq q q q 15 µ 20 The Generalized Linear Model Tackling the mean model problem Let E (Yi | xi ) ≡ µi be a function of the linear predictor xi β : µi = f (xi β ) It is more common to write the above as g (µi ) = xi β where g = f −1 is called the link function. The Generalized Linear Model The link function g is usually chosen so that g (µi ) can take on values in (−∞, +∞): • Bernoulli (binary) data: g (µ) = logit(µ) = log µ 1−µ • Poisson (count) data: g (µ) = log(µ) Aside: Any (invertible) function g can be chosen as the link, but a small number of choices (esp. log, logistic) predominate for various mathematical and practical reasons. The Generalized Linear Model Tackling the variance problem Specify a variance function relating the variance of the outcome to the mean: Var (Yi | xi ) = V (µi ) Examples: • Bernoulli (binary) data: V (µ) = µ(1 − µ) • Poisson (count) data: V (µ) = µ The Generalized Linear Model The combination of 1 2 Letting the mean be a function of a linear predictor Letting the variance be a function of the mean constitutes the Generalized Linear Model (GLM). Note: In the GLM, we do not explicitly specify the distribution of the outcome, but the choice of link and variance functions g and V often specify it implicitly. GLMs: Coefficient interpretation In GLMs, coefficient interpretation is similar to the linear model but must take into account the link function g : “A one-unit increase in X is associated with a β -unit increase/decrease in g (µ)” Can work backwards to get familiar cases: • Poisson regression: “A one-unit increase in X is associated with an exp(β ) percent increase/decrease in the mean of Y ” • Logistic regression: “A one-unit increase in X is associated with an expit(β ) percent increase/decrease in the odds of Y ” ...
View Full Document

Ask a homework question - tutors are online