This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: Penalized least squares versus generalized least squares representations of linear mixed models Douglas Bates Department of Statistics University of Wisconsin – Madison May 5, 2009 Abstract The methods in the lme4 package for R for fitting linear mixed models are based on sparse matrix methods, especially the Cholesky decomposition of sparse positive-semidefinite matrices, in a penalized least squares representation of the conditional model for the response given the random effects. The representation is similar to that in Hen- derson’s mixed-model equations. An alternative representation of the calculations is as a generalized least squares problem. We describe the two representations, show the equivalence of the two representations and explain why we feel that the penalized least squares approach is more versatile and more computationally efficient. 1 Definition of the model We consider linear mixed models in which the random effects are represented by a q-dimensional random vector, B , and the response is represented by an n-dimensional random vector, Y . We observe a value, y , of the response. The random effects are unobserved. For our purposes, we will assume a “spherical” multivariate normal condi- tional distribution of Y , given B . That is, we assume the variance-covariance matrix of Y | B is simply σ 2 I n , where I n denotes the identity matrix of order n . (The term “spherical” refers to the fact that contours of the conditional density are concentric spheres.) 1 The conditional mean, E[ Y | B = b ], is a linear function of b and the p-dimensional fixed-effects parameter, β , E[ Y | B = b ] = Xβ + Zb , (1) where X and Z are known model matrices of sizes n × p and n × q , respectively. Thus Y | B ∼ N ( Xβ + Zb , σ 2 I n ) . (2) The marginal distribution of the random effects B ∼ N ( , σ 2 Σ ( θ ) ) (3) is also multivariate normal, with mean and variance-covariance matrix σ 2 Σ ( θ ). The scalar, σ 2 , in (3) is the same as the σ 2 in (2). As described in the next section, the relative variance-covariance matrix, Σ ( θ ), is a q × q positive semidefinite matrix depending on a parameter vector, θ . Typically the dimension of θ is much, much smaller than q . 1.1 Variance-covariance of the random effects The relative variance-covariance matrix, Σ ( θ ), must be symmetric and pos- itive semidefinite (i.e. x Σ x ≥ , ∀ x ∈ R q ). Because the estimate of a vari- ance component can be zero, it is important to allow for a semidefinite Σ ....
View Full Document
- Fall '08