Register now to access 7 million high quality study materials (What's Course Hero?) Course Hero is the premier provider of high quality online educational resources. With millions of study documents, online tutors, digital flashcards and free courseware, Course Hero is helping students learn more efficiently and effectively. Whether you're interested in exploring new subjects or mastering key topics for your next exam, Course Hero has the tools you need to achieve your goals.

12 Pages

module-2

Course: ST 02, Fall 2009
School: CSU Channel Islands
Rating:

Word Count: 2619

Document Preview

of Department Statistics ST02: Multivariate Data Analysis and Chemometrics Bent Jrgensen and Yuri Goegebeur Module 2: Matrix algebra 2.1 2.2 2.3 2.4 2.5 2.6 2.7 Matrices, vectors and scalars . . . . . . . . . . Special matrices . . . . . . . . . . . . . . . . . Matrix and vector operations . . . . . . . . . . Matrix multiplication . . . . . . . . . . . . . . . Norm and trace . . . . . . . . . . . . . . . . . ....

Register Now

Unformatted Document Excerpt

Coursehero >> California >> CSU Channel Islands >> ST 02

Course Hero has millions of student submitted documents similar to the one
below including study guides, practice problems, reference materials, practice exams, textbook help and tutor support.

Course Hero has millions of student submitted documents similar to the one below including study guides, practice problems, reference materials, practice exams, textbook help and tutor support.
of Department Statistics ST02: Multivariate Data Analysis and Chemometrics Bent Jrgensen and Yuri Goegebeur Module 2: Matrix algebra 2.1 2.2 2.3 2.4 2.5 2.6 2.7 Matrices, vectors and scalars . . . . . . . . . . Special matrices . . . . . . . . . . . . . . . . . Matrix and vector operations . . . . . . . . . . Matrix multiplication . . . . . . . . . . . . . . . Norm and trace . . . . . . . . . . . . . . . . . . Orthogonal vectors, matrices and projection . . Eigenvalues, determinant and condition number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 . 2 . 3 . 5 . 8 . 8 . 10 2.1 Matrices, vectors and scalars Ugarte: You despise me, dont you? Rick Blaine: If I gave you any thought I probably would. [Casablanca, 1942] The following quote serves as a suitable introduction to this module: Linear algebra is the language of chemometrics. One cannot expect to truly understand most chemometric techniques without a basic understanding of linear algebra. This [module] reviews the basics of linear algebra and provides the reader with the foundations required for understanding most chemometrics literature. It is presented in a rather dense fashion. [...] The goal has been to condense into as few pages as possible the aspects of linear algebra used in most chemometrics methods. [Wise and Gallagher, 1998] Note that in this module, we use a mixture of the general notation introduced in Module 1, and a local notation, where letters are used freely to denote dierent vectors and matrices etc. A single number is usually called a scalar, and is represented by math italics, e.g. c. A matrix is a two dimensional array of numbers and is represented by bold upper case math italics e.g. X. For example X= 12 3 8 2.3 4 1 is a 23 matrix, having 2 rows and 3 columns. The dimensions of a matrix are normally presented with the number of rows rst and the number of columns second. We use square brackets to represent a matrix, in order to emphasize its rectangular shape. http://statmaster.sdu.dk/courses/ST02 February 7, 2006 2.2 Special matrices 2 Indices: The entries of a matrix are denoted by the same letter as the matrix, but in lower case math italics with two indices, e.g. X = {xij } for i = 1, . . . , 2 (rows) and j = 1, . . . , 3 (columns). For example, in the matrix X above, the (2,1) element is x21 = 2.3. A vector consists of a row or column of numbers and is represented by bold lower case italics e.g. x. Vectors can be considered as matrices with one dimension equal to 1. For example z = 3 14 9.4 0 is a 1 4 row vector (sometimes the entries are separated by commas), and 5.6 w = 2.8 1.9 a 3 1 column vector. 2.2 Special matrices A square matrix is one that has the same number of rows and columns, e.g. 7 4 2 A = 11 3 4.1 3 4 12 is a 3 3 square matrix. The main diagonal of a square matrix consists of the numbers aii with equal row and column indices, and is usually represented as a vector. So 7 3 12 is the row vector representing the main diagonal of the matrix A from above. A square matrix is called diagonal when all the non-diagonal entries are zero, e.g. A= 1 0 0 4 . We let diag{a1 , . . ., an } denote the diagonal matrix with main diagonal entries a1 , . . ., an . We let diagA denote the row vector containing the diagonal of A. An identity matrix (sometimes called a unit matrix) is a diagonal matrix whose main diagonal elements are equal to 1. It is denoted by I. For example I= 1 0 0 1 is an identity matrix. A subscript may be added to emphasize the dimension of the matrix, so I n denotes the n n identity matrix. http://statmaster.sdu.dk/courses/ST02 February 7, 2006 2.3 Matrix and vector operations 3 A zero matrix or vector is any matrix or vector whose entries are all zero. For example 0 0 0= 0 0 0 0 is the 3 2 zero matrix, which may also be denoted 032 to emphasize its dimensions. In particular, we use 0 to denote the origin of a vector space. vector whose entries are all 1, e.g. 1 1 1 A One matrix or vector is any matrix or 1 1= 1 1 Again its dimensions may be added as a subscript for emphasis, e.g. 132 . 2.3 Matrix and vector operations Addition and subtraction: Two matrices can be added or subtracted componentwise if they have identical dimensions (are conformable for addition). Hence 9 7 0 7 9 0 8 4 + 11 3 = 19 7 2 4 5 6 3 10 and 9 7 0 7 9 14 8 4 11 3 = 3 1 2 4 5 6 7 2 Scalar multiplication: A matrix can be multiplied by a constant, which is done componentwise. For example, if X= then the scalar multiple 3X is 3X = 36 9 24 6.9 12 3 12 3 8 2.3 4 1 The transpose X of a matrix X is the matrix formed by letting columns become rows (or rows columns), and is denoted by the superscript . So, for example, the 4 3 matrix 3.1 0.2 6.1 4.8 5 13.8 X = 20 1 2.2 5.1 3.8 3.94 http://statmaster.sdu.dk/courses/ST02 February 7, 2006 2.3 Matrix and vector operations 4 becomes a 3 4 matrix after transposition, 3.1 4.8 20 5.1 X = 0.2 5 1 3.8 . 6.1 13.8 2.2 3.94 Some authors use the superscript instead of for transpose. A square matrix A is called symmetric if A = A , e.g. A= 1 2 2 4 . Linear combinations: If a1 , . . . , ak are vectors of common dimension, and c1 , . . . , ck are constants, then the vector c1 a1 + + ck ak is called a linear combination of a1 , . . . , ak with coecients c1 , . . . , ck . The linear subspace spanned by a1 , . . . , ak is the set of all linear combinations of a1 , . . . , ak with arbitrary c1 , . . . , ck . Vectors a1 , . . . , ak are said to be linearly independent if c1 a1 + + ck ak = 0 implies c1 = = ck = 0. The rank of a matrix is the maximum number of linearly independent columns of the matrix. For example 1 2 = 2, rank 3 4 but rank because the vectors 1 2 3 6 = 1. 1 2 and are linearly dependent. The rank is also equal to 3 6 the maximum number of linearly independent rows of the matrix. Geometric interpretation: consider for example the three columns x1 , x2 , x3 of the above matrix X. They are 4-vectors. If they all point in the same direction (up to a sign), then the rank of X is 1. If this is not the case, but they all lie within a plane through 0, then X has rank 2. If this is not the case, then the rank is 3. The maximum possible rank for a matrix is the smallest of its column and row dimensions. In case of maximum rank the matrix is said to have full rank, or be nonsingular. A singular matrix is also called rank-decient, or is said to have a problem of multicollinearity. http://statmaster.sdu.dk/courses/ST02 February 7, 2006 2.4 Matrix multiplication 5 2.4 Matrix multiplication If matrix A = {aij } has dimensions k m and matrix B = {bij } has dimensions m n (having the second dimension of A and the rst dimension of B in common), they are said to be comformable for multiplication. Then the matrix product C = {ci } is a matrix of dimensions k n with elements dened by m ci = j=1 aij bj and we write C = AB. Example 1 7 9 3 2 5 6 17 67 38 = 54 93 123 42 12 25 62 31 6 10 11 3 0 1 8 5 Example: Consider the Beer-Lambert law for m constituents and k wavelengths plus noise: m xj = =1 y a j + ej , for wavelengths numbered j = 1, . . . , k. This equation may be written in matrix notation as follows: x = yA + e, where x = [x1 . . . xk ] is 1 k, A = {a j } is m k, y = [y1 . . . ym ] is 1 m, e = [e1 . . . ek ] is 1 k. In particular, the matrix product of two square matrices of common dimension is a square matrix of that dimension. Multiplication of matrices is not commutative, that is, generally AB = BA, even if the second product is allowable. Associative rule for matrix product:. (AB)C = A(BC) = ABC where the last form may be used due to the equality of the two rst. Distributive rule of matrix multiplication and addition: A(B + C) = AB + AC http://statmaster.sdu.dk/courses/ST02 7, February 2006 2.4 Matrix multiplication 6 Let a and b be column vectors of common dimension. The inner product (dot product) of a and b is dened by ab=a b = b a The result is a scalar. Note that each element in the matrix AB is the result of an 9 be inner product between a row of A and a column of B. For example, let a = 3 11 the second row of the rst matrix above (represented as a column) and b = be 8 the third column of the second matrix above. Then the (2,3) entry of the product is a b = 9 11 + 3 8 = 123 An n 1 one vector 1 is also known as a summing vector, because 1 x is the sum of the elements of x. Let a and b be column vectors of the dimensions m and n, respectively. The outer product of a and b is the m n matrix dened by ab = {ai bj } It may be thought of as the matrix product of a column vector and a row vector, which are always conformable for multiplication, because they have the dimension 1 in common. Inverse matrix: A non-singular square matrix A has a unique inverse matrix A1 (square with the same dimension as A), which satises AA1 = A1 A = I. A is then said to be invertible. See Examples for various illustrations of how the inverse of a matrix is calculated. Inverse of a product: For A and B invertible square matrices of the same size: (AB)1 = B 1 A1 Transpose of product: For A and B conformable for multiplication (AB) = B A Matrix inverse and transpose commute: A 1 = A1 = A where the last notation may be used for convenience to represent either of the two forms. http://statmaster.sdu.dk/courses/ST02 February 7, 2006 2.4 Matrix multiplication 7 Linear equations: For an invertible square matrix A and given vector b, the equation Ax = b with x unknown, has unique solution x = A1 b. For example, for a 3 3 matrix, this equation is equivalent to the following system of linear equations in x1 , x2 , x3 : a11 x1 + a12 x2 + a13 x3 = b1 a21 x1 + a22 x2 + a23 x3 = b2 a31 x1 + a32 x2 + a33 x3 = b3 If A is square, but not invertible, then any matrix A that satises AA A = A is called a generalized inverse of A. There are in general many such matrices A . The Moore-Penrose inverse A (unique) is further assumed to satisfy A AA = A AA A A = AA = A A Let A be an n k matrix of rank k n. Then the k n matrix A+ = A A 1 A is called the left pseudo-inverse of A. Note that the rank condition on A guarantees that the k k matrix A A is invertible. A+ satises the condition A+ A = I k Similarly, let A be an n k matrix of rank n k. Then the k n matrix A =A AA 1 is called the right pseudo-inverse of A. Then A satises the condition AA = I n http://statmaster.sdu.dk/courses/ST02 February 7, 2006 2.5 Norm and trace 8 2.5 Norm and trace Vector norm (length): Consider the n-vector x1 . x = . . . xn The norm of x is dened by x = = x x x2 + + x2 . n 1 The norm of x is the length of the line going from 0 to the point in n-space indicated by x. A vector of norm 1 is called a unit vector. The unit vector in the direction of x is dened by standardization x . e= x The trace tr of a square matrix is the sum of its diagonal elements, e.g. tr 1 2 3 4 = 1 + 4 = 5. Trace operator satises a kind of commutative property: tr(AB) = tr(BA) for matrices A and B such that AB and BA both exist (i.e. A and B dimension). have common 2.6 Orthogonal vectors, matrices and projection The two n-vectors x and y are said to be orthogonal if their inner product is zero, x y = 0. For example, the matrix has orthogonal columns. A set of vectors is said to be an orthogonal set if the vectors are pairwise orthogonal. An orthogonal set consisting of unit vectors, is called an orthonormal set. http://statmaster.sdu.dk/courses/ST02 February 7, 2006 1 1 0 5 1 1 2.6 Orthogonal vectors, matrices and projection 9 If X Y =0 the columns of X are all orthogonal to those of Y . A matrix A is called orthogonal if the columns of A are orthonormal, that is, A A=I It would have been more logical to call A an orthonormal matrix, but the above terminology is standard in linear algebra. For example, the matrix 0.7071 0.1925 0 0.9623 0.7071 0.1925 is orthogonal. If a square matrix A is orthogonal, then so is A , and A1 = A . The orthogonal projection of the n-vector y onto the linear subspace spanned by the columns of the n k matrix X is Hy, where H=X X X which requires that rank X = k. Note the following useful properties of rank: rank X = rank X X = rank XX . 1 X Matrix H is a projection matrix, also known in statistics as a hat matrix. It is symmetric and idempotent, that is, it satises H 2 = HH = H. Its trace is tr(H) = k. In the simplest case, the projection of y on x is e y e, where e = x x is the unit vector in the direction of x. http://statmaster.sdu.dk/courses/ST02 February 7, 2006 2.7 Eigenvalues, determin...

Find millions of documents on Course Hero - Study Guides, Lecture Notes, Reference Materials, Practice Exams and more. Course Hero has millions of course specific materials providing students with the best way to expand their education.

Below is a small sample set of documents:

CSU Channel Islands - ST - 02
Department of Statistics ST02: Multivariate Data Analysis and ChemometricsBent Jrgensen and Yuri GoegebeurST02: Multivariate Data Analysis and Chemometrics Preface Introduction Module 1: Chemometrics and NIR spectroscopy Module 2: Matrix alge
UCSD - CUNY - 2007
Multiple That: A strategy for reducing integration costs Laura Staum and Ivan Sag (Stanford University) lstaum@stanford.edu When people encounter the word that in a sentence like (1), they formulate expectations about what will come next. Once they k
UCSD - CUNY - 2007
Visiting relatives with ERPs can be instructive: Subject/object processing asymmetries in Korean relative clauses Nayoung Kwon, Robert Kluender, Maria Polinsky &amp; Marta Kutas (University of California, San Diego) nayoung@ling.ucsd.edu In English, (A)
UCSD - CUNY - 2007
Scrambling and information structure in Korean sentence comprehension Kyuseek Hwang &amp; Amy J. Schafer (University of Hawai'i at Mnoa) kyuseek@hawaii.edu Flexible word-order languages such as Japanese and Finnish allow every maximal phrase to &quot;scramble
UCSD - CUNY - 2007
20th annual CUNY conference on human sentence processingMarch 29-31, 2007, La Jolla, California20th annual C U N Y conference on human sentence processingMarch 29-31, 2007, La Jolla, California hosted by the Center for Research in Language at the
UCSD - HDP - 121
Embodiment: The Mind in the BodyNotice: My office hours M T 1-2 pm How to build a baby:Version 1.0: The clerical baby Repository of rules; filing cabinet of facts Version 2.0: Fully-loaded; low mileage.Rules that babies look by:Condition:Alert,
UCSD - HDP - 1
Name (last,first):_PID:_ Section #: A_ Day &amp; time:_ TA name:_ HDP 1 MIDTERM Fall 2005 Exam Form B When you get this exam, be sure to fill out the identifying information at the top of each page. When you are finished, review your answers, make sur
CSU Channel Islands - BIOD - 145
BioSci D145 Lecture #1 Bruce Blumberg (blumberg@uci.edu) 4103 Nat Sci 2 - office hours Tu, Th 3:30-5:00 (or by appointment) phone 824-8573 TA Stephanie Casey (scasey@uci.edu) 4351 Nat Sci 2, 824-6873, 3116 check e-mail and noteboard daily for
CSU Channel Islands - NUFACT - 06
Searching for new physics with neutrinosPatrick Huber University of Wisconsin, MadisonNuFact 06 Workshop August 24-30, 2006 University of California, IrvineP. Huber p.1/31Outline Why neutrinos? Neutrino oscillations New physics in os
CSU Channel Islands - NUFACT - 06
Neutrino Scattering Experiments: What do oscillation experiments need?NuFact06 Deborah Harris Fermilab August 27, 2006August 27, 2006Deborah Harris What do Oscillation Experiments Need?1Even the APS knows what we need.The recent APS Multidi
CSU Channel Islands - NUFACT - 06
Physics of Massive Neutrinos: Lecture IIMu-Chun Chen Fermilab (After Jan 2007: UC-Irvine)NuFact 2006 Summer School, UCLA/UC Irvine, August 15-17, 2006The Plan Lecture I: History Neutrinos in the Standard Model Quantum mechanics of oscill
CSU Channel Islands - NUFACT - 06
Physics of Massive Neutrinos: Lecture IIIMu-Chun Chen Fermilab (After Jan 2007: UC Irvine)NuFact 2006 Summer School, UCLA/UC Irvine, August 15-17, 2006Terrestrial Neutrinos man made from accelerator and nuclear reactors different energies
CSU Channel Islands - NUFACT - 06
Fermilab-Conf-06-237-AD August 10, 2006Accelerator-Based Neutrino BeamsSacha E. Kopp, Department of Physics, University of Texas at AustinContents1 Introduction 2 Accelerator Neutrino Beam Concept 2 33 Production of Hadrons in the Target 6 3.
CSU Channel Islands - NUFACT - 06
Physics of Neutrino InteractionsH. Gallagher NuFact Summer School Aug 15-16, 2006 UCLAIn these two lectures I will cover a range of topics on the interactions of neutrinos with matter, focusing on interactions in the energy range of interest to a
CSU Channel Islands - ICLASS - 2009
Exhibitor Application Form Yes, I would like to exhibit at ICLASS 2009. No, I will not be exhibiting this year, but keep me on the mailing list. Company: _ Address: __ City: _ State, Zip Code: _Province, Postal Code: _ Contact Name: __ Email: _B
CSU Channel Islands - ICLASS - 2009
Dear Potential Exhibitor: On behalf of the ICLASS 2009 organizing committee, I would like to invite your company to be an exhibitor at the upcoming 11th International Conference on Liquid Atomization and Spray Systems. The conference will be held in
CSU Channel Islands - ICLASS - 2009
ILASS AmericasInstitute for Liquid Atomization and Spray Systems North and South AmericaDear Prospective ICLASS-2009 Sponsor: The 2009 ICLASS (11th International Conference on Liquid Atomization and Spray Systems) conference will be held July 26
CSU Channel Islands - ICLASS - 2009
REGISTRATION FORM First Name Last Name Name on badge Company/Organization Address Line 1 Address Line 2 City US State/Canadian Province Int'l State/County/Province (non US/Canada) Zip (Postal Code) Country Work Phone email Special R
CSU Channel Islands - ICLASS - 2009
Companies Invited to Exhibit at ICLASS 2009ANSYS/Fluent Artium Technologies BETE Fog Nozzle, Inc. CD-adapco Clean Air Systems Coen Co., Inc. The Cooke Corporation Dantec Dynamics Inc. Delavan Gas Turbine Products Dow Chemical Co. Eastman Kodak Co. E
CSU Channel Islands - P - 231
Class ProjectsGoals1. Practice designing and implementing an object oriented program. 2. Do something interesting and potentially useful to others. 3. Look at what other people did by refereeing other projects.Desirable Properties1. 2. 3. 4. 5.
CSU Channel Islands - P - 214
Formula Sheet ln n! n ln n n Distributions Binomial distribution Gaussian Distribution Poisson Distribution P (n, N ) = N! pn q N n n!(N n)! 1 2 2 e(x) /2 dx 2 = Np (2) (3) (4) (1)P (x)dx = P (n) =n e n! Thermodynamics E = Q WdW = pdV =
CSU Channel Islands - P - 115
Formula Sheetln n! n ln n n rn =n=0 1 1r(1) 3/2 2 (2)ex2 dx = 1/2ex2x dx =2Distributions Binomial distribution Gaussian Distribution P (n, N ) = N! pn q N n n!(N n)! 1 2 2 e(x) /2 dx 2 = Np (3) (4) (5)P (x)dx =
CSU Channel Islands - P - 3
w uaw w s h wak g a aeaw s r aw s yae w u ` i qqXcto2gAqqoqtxc2xccd\$oqq2dAAxctA5bApqtp 27qqsdqtdqXcdigqhqXvsAAq22pt\$9evqqoqtxxdAhbvp5ctesdqti gi s i w i w u aw w g a w s h b Y w s h e a ub ` w a i w w uaw w
CSU Channel Islands - P - 3
3 v (3 ( 3U 1 )U13 ) &amp; ( v &amp;t p d ka)u}01#2#4u}a3Q6jbAb1 ) &amp; pt 1 )U &amp; p d1 &amp;3 3 v (3 v &amp;t p d1 3U 1 )U13 ) &amp; ( p v \$ ( tp1U x &amp; 1 ) &amp; uX&amp;2#qq4u'ka)um6jbb#kg6uqa3#wUua3bgh#kqgU p v &amp;t p d1 p v \$ ( tp1U x p d1 &amp; 1 d (p x p d1 &amp;1 vU qp ( p
CSU Channel Islands - P - 238
p u tx u { lv s xp put rx xxp x vu s tux u w x v t t r Ie8x8Sdm i SzIqeuzU8qS&quot;jsIapIee}sj~vcSIISqdIIIS rx wv s rx pu { r l uvxt{ l u sxt x x SIvuIIIqSedIGSjA&quot;sqsUxcSddjrjAvjSepedS8yu sSs4sdIeIee8jjs~u|8q8qwpqsSeSI|pISeI&quot;uS
CSU Channel Islands - P - 3
Professor Yu Seating Chart Student# 00602614 10273538 10376715 10541468 10606691 11069112 11162038 11564037 12011285 12991855 13022659 13362153 13563460 14021836 14056254 14717287 15238504 15678818 15908326 16085675 16114295 16168124 16431041 1668237
CSU Channel Islands - ICS - 161
ICS 161 Algorithms Spring 2005 Final Exam Please answer the following eight questions on the answer sheets provided. Answers written on other pages or on the wrong sheet will not be scored. Be sure to write your name and student ID on all three an
CSU Channel Islands - ICS - 260
ICS 260 Fall 2001 Second MidtermName: Student ID:1: 2: 3: 4: 5:Total:1. Longest symmetric subsequence. (20 points) A symmetric sequence or palindrome is a sequence of characters that is equal to its own reversal; e.g. the letters in the phr
CSU Channel Islands - ICS - 261
ICS 261-Sample solutions on LCAs and augmented tree structuresDavid Eppstein, ICS, UC Irvine March 4, 19991. Suppose that instead of least common ancestor queries in a tree, we only need to answer ancestordescendant queries: is node x an ancestor o
CSU Channel Islands - ICS - 261
ICS 261-Sample problems on LCAs and augmented tree structuresDavid Eppstein, ICS, UC Irvine February 25, 19991. Suppose that instead of least common ancestor queries in a tree, we only need to answer ancestordescendant queries: is node x an ancest
CSU Channel Islands - ICS - 261
ICS 261-Sample problems on segment and interval treesDavid Eppstein, ICS, UC Irvine March 9, 19991. The &quot;Manhattan Skyline&quot; problem: suppose we are given a collection of rectangles, each of them having their bottom edge on the x axis, as in the fig