This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: Matrix algebra primer, page 1 M ATRIX A LGEBRA FOR S TATISTICS : P ART 1 Matrices provide a compact notation for expressing systems of equations or variables. For instance, a linear function might be written as: y = x 1 b 1 + x 2 b 2 + x 3 b 3 + + x k b k This is really the product of a bunch of b variables and a bunch of x variables. A vector is simply a collection of variables ( in a particular order ) . We could define a ( k-dimensional ) vector x = ( x 1 , x 2 , , x n ) , and another vector b = ( b 1 , b 2 , , b n ) . Again, these vectors simply represent the collections of x and a variables; the dimension of the vector is the number of elements in it. We define the product of two vectors to be: x ! b " x i i = 1 k # b i = x 1 b 1 + x 2 b 2 + + x k b k ( Specifically, this is called a dot product or inner product ; there exist other ways to calculate products, but we won t be using those. ) If you think of b as the collection of all b variables and x as the collection of all x variables , then the product x ! b is the product of each b variable with the corresponding x variable. You can calculate the ( dot ) product only when the two vectors have the same dimension. Example: Let a = (1,2,3,4) and let b = (5,6,7,8) . These are both 4-dimensional vectors, so we can calculate their dot product. a ! b = " i = 1 4 a i b i = (1 ! 5) + (2 ! 6) + (3 ! 7) + (4 ! 8) = 5 + 12 + 21 + 32 = 70 . Sometimes, we say that two vectors are orthogonal if their dot product equals zero. Orthogonality has two interpretations. Graphically, it means that the vectors are perpendicular. On a deeper philosophical level, it means that the vectors are unrelated. Example: Let c = (0,1) and let d = (1,0) . Since c ! d = (0 ! 1) + (1 ! 0) = + = , the vectors are orthogonal. Graphically, we can represent c as a line from the origin to the point (0,1) and d as a line from the origin to (1,0) . These lines are perpendicular. ( On a deeper sense, they are unrelated because the first vector moves only along the x-axis and never changes its y-direction; the second moves only along the y-axis and doesn t change its x-direction. ) Example: Let e = (1,1) and let f = (1, ! 1) . Since e ! f = (1 ! 1) + (1 ! " 1) = 1 + ( " 1) = , the vectors are orthogonal. Again, we can show that these lines are perpendicular in a graph. ( It s a bit hard to graph how they are unrelated; but we could create a new coordinate system for the space in which they are. ) There s a moral to this exercise: two vectors can have a product of zero, even though neither of the vectors is zero. Matrix algebra primer, page 2 Finally, dot products have a statistical interpretation. Let s let x and y be two random variables, each with mean zero. We will collect a sample of size N , and we will record the value of x i and y i for each observation. We can then construct a vector x = ( x 1 , x 2 , , x N ) and a similar vector y = ( y 1...
View Full Document
- Spring '08