Regression.pdf

# Random vectors let y y 1 y 2 y n be a random vector

• 63

This preview shows pages 10–13. Sign up to view the full content.

RANDOM VECTORS Let Y = ( Y 1 , Y 2 , ..., Y n ) be a random vector with the probability density function (pdf) denoted by f ( y ) (describing how Y 1 , Y 2 , ..., Y n are jointly distributed). Then, E( Y ) = ( E ( Y 1 ) , E ( Y 2 ) , ..., E ( Y n )) and the variance-covariance matrix of Y is Var( Y 1 ) Cov( Y 1 , Y 2 ) · · · Cov( Y 1 , Y n ) Cov( Y 2 , Y 1 ) Var( Y 2 ) · · · Cov( Y 2 , Y n ) . . . . . . . . . Cov( Y n , Y 1 ) Cov( Y n , Y 2 ) · · · Var( Y n ) PAGE 10

This preview has intentionally blurred sections. Sign up to view the full version.

1.3 Matrix approach to Simple Linear Models c circlecopyrt HYON-JUNG KIM, 2017 defined by E[( Y E ( Y ))( Y E ( Y )) ] = Var( Y ). Since Cov( Y i , Y j ) = Cov( Y j , Y i ), it follows that the variance-covariance matrix is symmetric. The covariance of two random vectors, Y n × 1 and Z m × 1 is given by Cov( Y , Z ) = Cov( Y 1 , Z 1 ) Cov( Y 1 , Z 2 ) · · · Cov( Y 1 , Z m ) Cov( Y 2 , Z 1 ) Cov( Y 1 , Z 2 ) · · · Cov( Y 2 , Z m ) . . . . . . . . . Cov( Y n , Z 1 ) Cov( Y n , Z 2 ) · · · Cov( Y n , Z m ) n × m . BASIC RESULTS Suppose that A, B are n × m matrices of constants and that c is a vector of constants. Let V be the variance-covariance matrix of Y . i) E( A Y ) = A E( Y ) ii) Var( A Y + c ) = A Var( Y ) A = AV A . iii) Cov( A Y , B Z ) = A Cov( Y , Z ) B iv) Y A Y = ∑ ∑ a ij Y i Y j is a quadratic form and E ( Y A Y ) = µ A µ + tr( AV ) , where E( Y ) = µ and tr( W ) = n i =1 w ii is the trace of W n × n . Recall that expectation of a linear combination of random variables, U k , k = 1 , · · · , m : E( m summationdisplay k =1 a k U k ) = m summationdisplay k =1 a k E(U k ) The variance of a linear combination of random variables is Var( m summationdisplay k =1 a k U k ) = m summationdisplay k =1 m summationdisplay l =1 a k a l Cov(U k , U l ) = m summationdisplay k =1 a 2 k Var(U k ) + summationdisplay summationdisplay l negationslash = k a k a l Cov(U k , U l ) Definition: The random vector Y = ( Y 1 , Y 2 , ..., Y n ) is said to have a multivariate normal distribution with mean µ and variance-covariance matrix V if its (joint) probability density function is PAGE 11
1.3 Matrix approach to Simple Linear Models c circlecopyrt HYON-JUNG KIM, 2017 given by f Y ( y ) = 1 (2 π ) n/ 2 | V | 1 / 2 exp {− 1 2 ( y µ ) V 1 ( y µ ) } , for all y R n . Shorthand notation for this statement is Y MVN( µ , V ) . If Y MVN( µ , V ), then marginally, each Y i Normal( µ i , Var( Y i )). If A is a matrix of constants and c is a vector of constants, A Y + c MVN ( A µ + c , AV A ) . Regression Model in Matrix Notation For more complicated models it is useful to write the model in matrix notation. When we expand out the simple linear model, Y i = β 0 + β i X i + ǫ i , i = 1 , . . . , n Y 1 = β 0 + β 1 X 1 + ǫ 1 Y 2 = β 0 + β 1 X 2 + ǫ 2 . . . Y n = β 0 + β 1 X n + ǫ n This can be expressed as Y 1 Y 2 . . . Y n = 1 X 1 1 X 2 . . . . . . 1 X n bracketleftBigg β 0 β 1 bracketrightBigg + ǫ 1 ǫ 2 .

This preview has intentionally blurred sections. Sign up to view the full version.

This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern