day3lm - PADP 8130: Linear Models Math Review ...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: PADP 8130: Linear Models Math Review Angela Fer7g, Ph.D. Deriva7ve Rules Power Func7ons: Natural Log Func7ons: dn ( x ) = nx n−1 dx d 1 (log x ) = dx x Exponen7al Func7ons: dx (e ) = e x dx 1 Deriva7ve Rules Product Rule: d d d f ( x ) g( x ) = f ( x ) g( x ) + g( x ) f ( x ) dx dx dx Quo7ent Rule: d d d f ( x ) g( x ) dx f ( x ) − f ( x ) dx g( x ) = dx g( x ) g2 ( x ) Chain Rule: dz dz dy = where z = f ( y( x )) dx dy dx Matrix Algebra Matrix algebra is the algebra of arrays of numbers, which is useful for simplifying the descrip7on and handling of large amounts of data. ⎡ a11 a1k ⎢ A=⎢ ⎢ an1 ank ⎢ ⎣ ⎤ ⎥ ⎥ ⎥ ⎥ ⎦ 2 Terminology •  •  •  •  Row vector – 1xn matrix Column vector – nx1 matrix Square matrix – nxn matrix Types of square matrices: –  Symmetric: aik = aki –  Diagonal: nondiagonal elements are zero –  Scalar: diagonal matrix with same value for diagonal elements –  Iden7ty (I): diagonal matrix with 1’s down the diagonal –  Triangular: zeros above or below diagonal Algebraic Manipula7on of Matrices 1 Transpose: B = A ' iff bij = a ji , ∀i, j –  Symmetric matrix: A = A ' –  Any matrix: A = A '' Trace (square matrix): sum of diagonal elements Addi7on: C = A + B = [ aij + bij ] –  Have to have same dimensions –  Rules: A + 0 = A A - B = [ aij − bij ] A+B = B+A (A + B)' = A' + B' 3 Algebraic Manipula7on of Matrices 2 Mul7plica7on: –  Inner product of 2 vectors: a'b = [ –  Product of 2 matrices: ⎡ b1 ⎤ ⎥ = a1b1 + a2 b2 a1 a2 ] ⎢ ⎢ b2 ⎥ ⎣ ⎦ ⎡ a11 a12 ⎤ ⎡ b11 b12 b13 ⎤ ⎥ ⎥⎢ AB = ⎢ ⎢ a21 a22 ⎥ ⎢ b21 b22 b23 ⎥ ⎣ ⎦⎣ ⎦ ⎡ a11b11 + a12 b21 a11b12 + a12 b22 a11b13 + a12 b23 =⎢ ⎢ a21b11 + a22 b21 a21b12 + a22 b22 a21b13 + a22 b23 ⎣ •  Have to be conformable: [nxk][kxj]=[nxj] ⎤ ⎥ ⎥ ⎦ Algebraic Manipula7on of Matrices Example Without matrix algebra, would write: yi = β1 xi1 + β2 xi 2 + β3 xi 3 + + β j xij + ε i for each observation i. Example of usefulness of matrices: y1 ⎤ ⎡ x11 x12 x1 j ⎥⎢ x2 j y2 ⎥ ⎢ x21 x22 =⎢ ⎥⎢ ⎥ yn ⎥ ⎢ xn1 xn 2 xnj ⎦⎣ [ nx1] = [ nxj ][ jx1] + [ nx1] y = Xb + e ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ ⎤⎡ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎦⎣ β1 ⎤ ⎡ ⎥⎢ β2 ⎥ ⎢ + ⎥⎢ ⎥⎢ βj ⎥ ⎢ ⎦⎣ ε1 ⎤ ⎥ ε2 ⎥ ⎥ ⎥ εn ⎥ ⎦ 4 Algebraic Manipula7on of Matrices 3 Sum of values: Let ι = vector of ones; x, y = vectors ∑x i = ι'x 1 ∑ xi = (ι 'ι )−1ι ' x n ∑ xi2 = x ' x x= ∑x y ii =x ' y Idempotent matrices •  Idempotent matrices (its square equals itself) are important in sta7s7cs 1 •  An important idempotent matrix: M o = I - n ii' •  Mo “demeans” a vector: M x = (I − 1 ιι ')x = x − 1 ιι ' x n n •  x M o x = ∑ ( xi − x )2 ' ⎛1 1⎞ ⎛ x ⎞ ⎜ n n ⎟⎛ ⎟⎜ ⎜ ⎟⎜ = ⎜ ⎟ −⎜ ⎟⎜ ⎜x ⎟ ⎜1 1 ⎟⎜ ⎝ ⎠⎜ ⎟⎝ n n O 1 n ⎝ ⎠ x1 ⎞ ⎟ ⎟ xn ⎟ ⎠ ⎛ x1 − x ⎞ ⎜ ⎟ =⎜ ⎟ ⎜ xn − x ⎟ ⎝ ⎠ 5 Other Special Matrices •  Orthogonal: A if AA ' = I •  Posi7ve Definite: A if x ' Ax > 0 •  Inverse: A −1 if A -1A = I Can Par77on Matrices ⎛ ⎜ A=⎜ ⎜ ⎜ ⎜ ⎝ a11 a12 a13 a21 a22 a23 a31 a32 a33 a 41 a42 a43 a14 ⎞ ⎟ a24 ⎟ ⎛ A11 =⎜ a34 ⎟ ⎝ A 21 ⎟ a 44 ⎟ ⎠ ⎛ a11 a12 ⎞ A12 ⎞ ⎟ where A11 = ⎜ ⎟ A 22 ⎠ ⎝ a21 a22 ⎠ 6 Determinants A determinant is a polynomial of the elements of a square matrix. a11 a12 a21 a22 a11 a12 a13 det A = A = a21 a22 a23 = a11 a31 a32 a33 det A = A = = a11a22 − a12 a21 a22 a23 a32 a33 a21 a23 a31 − a12 a33 + a13 a21 a22 a31 a32 Deriva7on of the Inverse Now that we know how to get the determinant, we can get the inverse. 2x2 case: −1 ⎛ a11 a12 ⎞ 1 ⎛ a22 − a12 ⎞ ⎜ ⎟= ⎜ ⎟ det A ⎝ − a21 a11 ⎠ ⎝ a21 a22 ⎠ Diagonal case: ⎛1 ⎞ −1 0⎟ ⎜d ⎛d 0⎞ 11 ⎜ ⎝0 d22 ⎟ ⎠ =⎜ ⎜ ⎜ ⎝ 11 0 ⎟ 1⎟ ⎟ d22 ⎠ 7 Use of the inverse •  Cannot divide a matrix; instead mul7ply by its inverse •  Example: want to solve for X AX = B A -1AX = A -1B IX = A -1B X = A -1B •  Important note: X can only be solved for if the inverse of A exists and the inverse of A exists if det A is nonzero. Rank of a matrix: How we know if det A is nonzero •  The rank of a matrix is the number of linearly independent rows (and columns) in the matrix. –  If Xa=0 only for a=0, then X the columns/rows of X are linearly independent vectors. –  Said differently, a set of vectors are linearly dependent if one of the vectors can be wriien as a linear combina7on of the others. •  Matrix A has full rank if A is an mxn matrix where m≤n and the rank of A is m. The determinant of a matrix is non- zero iff it has full rank. 8 Eigenvalues •  Want to solve for a vector u (eigenvector) and a scalar λ (eigenvalue) such that Au=λu. •  The characteris7c equa7on is |A- λI|=0. Solve this equa7on for λ. •  One use of eigenvalues: the rank of a symmetric matrix if the number of non- zero eigenvalues. Puong it together # of non- zero eigenvalues = # rows of A A is full rank Determinant of A is non- zero Inverse of A exists! 9 ...
View Full Document

Ask a homework question - tutors are online