# lec 12-ppt.pdf - Lecture 12 Cauchy-Schwartz Inequality...

• 33

This preview shows page 1 - 12 out of 33 pages.

Lecture 12: Cauchy-Schwartz Inequality, Gram-Schmidt Process, QR Factorization and Orthogonal Transformations Longxiu Huang
Goal: Cauchy-Schwartz inequality. Angle between two vectors. Correlation coefficient. Gram-Schmidt process and QR factorization. Orthogonal transformations, orthogonal matrices, matrix transpose and the matrix of an orthogonal projection. 1 / 25
Table of Contents 1 Cauchy-Schwartz inequality 2 Gram-Schmidt Process 3 QR Factorization 4 Orthogonal Transformation 5 The Transpose of a Matrix 6 The Matrix of an Orthogonal Projection
Theorem (Pythagorean theorem) Let ~x, ~ y R n . The equation k ~x + ~ y k 2 = k ~x k 2 + k ~ y k 2 holds if (and only if) ~x and ~ y are orthogonal. 2 / 25
Theorem (An inequality for the magnitude of proj V ( ~x ) ) Consider a subspace V R n and a vector ~x R n . Then k proj V ~x k ≤ k ~x k . The statement is an equality iff ~x V . 3 / 25
Proof. Since for any vector ~x R n , we have ~x = ~x + proj V ( ~x ) . By Pythagorean theorem, we get k ~x k 2 = k ~x k 2 + k proj V ( ~x ) k 2 ≥ k proj V ( ~x ) k 2 and thereby k proj V ~x k ≤ k ~x k . 4 / 25
Theorem (Cauchy-Schwartz inequality)) If ~x and ~ y are vectors in R n , then | ~x · ~ y | ≤ k ~x kk ~ y k . The statement is an equality iff ~x and ~ y are parallel. 5 / 25
Definition (Angles between two vectors) Consider two nonzero vectors ~x and ~ y in R n . The angle θ between these vectors is defined as θ = arccos ~x · ~ y k ~x kk ~ y k . Note that θ [0 , π ] . 6 / 25
Table of Contents 1 Cauchy-Schwartz inequality 2 Gram-Schmidt Process 3 QR Factorization 4 Orthogonal Transformation 5 The Transpose of a Matrix 6 The Matrix of an Orthogonal Projection
Gram-Schmidt Process Consider a basis ~v 1 , . . . ,~v m of a subspace V of R n . Let V 1 = span ( ~v 1 ) V 2 = span ( ~v 1 ,~v 2 ) . . . V m = span ( ~v 1 , . . . ,~v m ) = V To find an orthonormal basis of V 1 , we normalize the basis vector ~v 1 and get ~u 1 = ~v 1 k ~v 1 k which forms an orthonormal basis of V 1 . 7 / 25
Gram-Schmidt Process continues... To find an orthonormal basis of V 2 , we have ~u 1 V 2 and look for a unit vector ~u 2 which is orthogonal to ~u 1 such that span ( ~u 1 , ~u 2 ) = V 2