la09-rips - LINEAR ALGEBRA W W L CHEN c W W L Chen, 1997,...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: LINEAR ALGEBRA W W L CHEN c W W L Chen, 1997, 2005. This chapter is available free to all individuals, on the understanding that it is not to be used for financial gain, and may be downloaded and/or photocopied, with or without permission from the author. However, this document may not be kept on any information storage and retrieval system without permission from the author, unless such system is not accessible to any individuals other than its owners. Chapter 9 REAL INNER PRODUCT SPACES 9.1. Euclidean Inner Products In this section, we consider vectors of the form u = (u1 , . . . , un ) in the euclidean space Rn . In particular, we shall generalize the concept of dot product, norm and distance, first developed for R2 and R3 in Chapter 4. Definition. Suppose that u = (u1 , . . . , un ) and v = (v1 , . . . , vn ) are vectors in Rn . The euclidean dot product of u and v is defined by u · v = u1 v1 + . . . + un vn , the euclidean norm of u is defined by u = (u · u)1/2 = (u2 + . . . + u2 )1/2 , 1 n and the euclidean distance between u and v is defined by d(u, v) = u − v = ((u1 − v1 )2 + . . . + (un − vn )2 )1/2 . PROPOSITION 9A. Suppose that u, v, w ∈ Rn and c ∈ R. Then (a) u · v = v · u; (b) u · (v + w) = (u · v) + (u · w); (c) c(u · v) = (cu) · v; and (d) u · u ≥ 0, and u · u = 0 if and only if u = 0. Chapter 9 : Real Inner Product Spaces page 1 of 15 c Linear Algebra W W L Chen, 1997, 2005 PROPOSITION 9B. (CAUCHY-SCHWARZ INEQUALITY) Suppose that u, v ∈ Rn . Then |u · v| ≤ u v. In other words, 2 2 |u1 v1 + . . . + un vn | ≤ (u2 + . . . + u2 )1/2 (v1 + . . . + vn )1/2 . 1 n PROPOSITION 9C. Suppose that u, v ∈ Rn and c ∈ R. Then (a) u ≥ 0; (b) u = 0 if and only if u = 0; (c) cu = |c| u ; and (d) u + v ≤ u + v . PROPOSITION 9D. Suppose that u, v, w ∈ Rn . Then (a) d(u, v) ≥ 0; (b) d(u, v) = 0 if and only if u = v; (c) d(u, v) = d(v, u); and (d) d(u, v) ≤ d(u, w) + d(w, v). Remark. Parts (d) of Propositions 9C and 9D are known as the Triangle inequality. In R2 and R3 , we say that two non-zero vectors are perpendicular if their dot product is zero. We now generalize this idea to vectors in Rn . Definition. Two vectors u, v ∈ Rn are said to be orthogonal if u · v = 0. Example 9.1.1. Suppose that u, v ∈ Rn are orthogonal. Then u+v 2 = (u + v) · (u + v) = u · u + 2u · v + v · v = u 2 + v 2. This is an extension of Pythagoras’s theorem. Remarks. (1) Suppose that we write u, v ∈ Rn as column matrices. Then u · v = vt u, where we use matrix multiplication on the right hand side. (2) Matrix multiplication can be described in terms of dot product. Suppose that A is an m × n matrix and B is an n × p matrix. If we let r1 , . . . , rm denote the vectors formed from the rows of A, and let c1 , . . . , cp denote the vectors formed from the columns of B , then r1 · c1 . AB = . . rm · c1 r1 · cp . . . . . . . rm · cp ... 9.2. Real Inner Products The purpose of this section and the next is to extend our discussion to define inner products in real vector spaces. We begin by giving a reminder of the basics of real vector spaces or vector spaces over R. Chapter 9 : Real Inner Product Spaces page 2 of 15 c Linear Algebra W W L Chen, 1997, 2005 Definition. A real vector space V is a set of objects, known as vectors, together with vector addition + and multiplication of vectors by elements of R, and satisfying the following properties: (VA1) For every u, v ∈ V , we have u + v ∈ V . (VA2) For every u, v, w ∈ V , we have u + (v + w) = (u + v) + w. (VA3) There exists an element 0 ∈ V such that for every u ∈ V , we have u + 0 = 0 + u = u. (VA4) For every u ∈ V , there exists −u ∈ V such that u + (−u) = 0. (VA5) For every u, v ∈ V , we have u + v = v + u. (SM1) For every c ∈ R and u ∈ V , we have cu ∈ V . (SM2) For every c ∈ R and u, v ∈ V , we have c(u + v) = cu + cv. (SM3) For every a, b ∈ R and u ∈ V , we have (a + b)u = au + bu. (SM4) For every a, b ∈ R and u ∈ V , we have (ab)u = a(bu). (SM5) For every u ∈ V , we have 1u = u. Remark. The elements a, b, c ∈ R discussed in (SM1)–(SM5) are known as scalars. Multiplication of vectors by elements of R is sometimes known as scalar multiplication. Definition. Suppose that V is a real vector space, and that W is a subset of V . Then we say that W is a subspace of V if W forms a real vector space under the vector addition and scalar multiplication defined in V . Remark. Suppose that V is a real vector space, and that W is a non-empty subset of V . Then W is a subspace of V if the following conditions are satisfied: (SP1) For every u, v ∈ W , we have u + v ∈ W . (SP2) For every c ∈ R and u ∈ W , we have cu ∈ W . The reader may refer to Chapter 5 for more details and examples. We are now in a position to define an inner product on a real vector space V . The following definition is motivated by Proposition 9A concerning the properties of the euclidean dot product in Rn . Definition. Suppose that V is a real vector space. By a real inner product on V , we mean a function , : V × V → R which satisfies the following conditions: (IP1) For every u, v ∈ V , we have u, v = v, u . (IP2) For every u, v, w ∈ V , we have u, v + w = u, v + u, w . (IP3) For every u, v ∈ V and c ∈ R, we have c u, v = cu, v . (IP4) For every u ∈ V , we have u, u ≥ 0, and u, u = 0 if and only if u = 0. Remarks. (1) The properties (IP1)–(IP4) describe respectively symmetry, additivity, homogeneity and positivity. (2) We sometimes simply refer to an inner product if we know that V is a real vector space. Definition. A real vector space with an inner product is called a real inner product space. Our next definition is a natural extension of the idea of euclidean norm and euclidean distance. Definition. Suppose that u and v are vectors in a real inner pruduct space V . Then the norm of u is defined by u = u, u 1 /2 , and the distance between u and v is defined by d(u, v) = u − v . Chapter 9 : Real Inner Product Spaces page 3 of 15 c Linear Algebra W W L Chen, 1997, 2005 Example 9.2.1. For u, v ∈ Rn , let u, v = u · v, the euclidean dot product discussed in the last section. This satisfies Proposition 9A and hence conditions (IP1)–(IP4). The inner product is known as the euclidean inner product in Rn . Example 9.2.2. Let w1 , . . . , wn be positive real numbers. For u = (u1 , . . . , un ) and v = (v1 , . . . , vn ) in Rn , let u, v = w1 u1 v1 + . . . + wn un vn . It is easy to check that conditions (IP1)–(IP4) are satisfied. This inner product is called a weighted euclidean inner product in Rn , and the positive real numbers w1 , . . . , wn are known as weights. The unit circle with respect to this inner product is given by {u ∈ Rn : u = 1} = {u ∈ Rn : u, u = 1} = {u ∈ Rn : w1 u2 + . . . + wn u2 = 1}. 1 n Example 9.2.3. Let A be a fixed invertible n × n matrix with real entries. For u, v ∈ Rn , interpreted as column matrices, let u, v = Au · Av, the euclidean dot product of the vectors Au and Av. It can be checked that conditions (IP1)–(IP4) are satisfied. This inner product is called the inner product generated by the matrix A. To check conditions (IP1)–(IP4), it is useful to note that u, v = (Av)t Au = vt At Au. Example 9.2.4. Consider the vector space M2,2 (R) of all 2 × 2 matrices with real entries. For matrices U= u11 u21 u12 u22 and V= v11 v21 v12 v22 in M2,2 (R), let U, V = u11 v11 + u12 v12 + u21 v21 + u22 v22 . It is easy to check that conditions (IP1)–(IP4) are satisfied. Example 9.2.5. Consider the vector space P2 of all polynomials with real coefficients and of degree at most 2. For polynomials p = p(x) = p0 + p1 x + p2 x2 and q = q (x) = q0 + q1 x + q2 x2 in P2 , let p, q = p0 q0 + p1 q1 + p2 q2 . It can be checked that conditions (IP1)–(IP4) are satisfied. Example 9.2.6. It is not difficult to show that C [a, b], the collection of all real valued functions continuous in the closed interval [a, b], forms a real vector space. We also know from the theory of real valued functions that functions continuous over a closed interval [a, b] are integrable over [a, b]. For f, g ∈ C [a, b], let b f, g = f (x)g (x) dx. a It can be checked that conditions (IP1)–(IP4) are satisfied. Chapter 9 : Real Inner Product Spaces page 4 of 15 c Linear Algebra W W L Chen, 1997, 2005 9.3. Angles and Orthogonality Recall that in R2 and R3 , we can actually define the euclidean dot product of two vectors u and v by the formula u·v = u (1) v cos θ, where θ is the angle between u and v. Indeed, this is the approach taken in Chapter 4, and the Cauchy-Schwarz inequality, as stated in Proposition 9B, follows immediately from (1), since | cos θ| ≤ 1. The picture is not so clear in the euclidean space Rn when n > 3, although the Cauchy-Schwarz inequality, as given by Proposition 9B, does allow us to recover a formula of the type (1). But then the number θ does not have a geometric interpretation. We now study the case of a real inner product space. Our first task is to establish a generalized version of Proposition 9B. PROPOSITION 9E. (CAUCHY-SCHWARZ INEQUALITY) Suppose that u and v are vectors in a real inner product space V . Then | u, v | ≤ u (2) v. Proof. Our proof here looks like a trick, but it works. Suppose that u and v are vectors in a real inner product space V . If u = 0, then since 0u = 0, it follows that u, v = 0, v = 0u, v = 0 u, v = 0, so that (2) is clearly satisfied. We may suppose therefore that u = 0, so that u, u = 0. For every real number t, it follows from (IP4) that tu + v, tu + v ≥ 0. Hence 0 ≤ tu + v, tu + v = t2 u, u + 2t u, v + v, v . Since u, u = 0, the right hand side is a quadratic polynomial in t. Since the inequality holds for every real number t, it follows that the quadratic polynomial t2 u, u + 2t u, v + v, v has either repeated roots or no real root, and so the discriminant is non-positive. In other words, we must have 0 ≥ (2 u, v )2 − 4 u, u v, v = 4 u, v 2 −4 u 2 v 2. The inequality (2) follows once again. Example 9.3.1. Note that Proposition 9B is a special case of Proposition 9E. In fact, Proposition 9B represents the Cauchy-Schwarz inequality for finite sums, that for u1 , . . . , un , v1 , . . . , vn ∈ R, we have n 1 /2 n ui vi ≤ i=1 i=1 1 /2 n u2 i 2 vi . i=1 Example 9.3.2. Applying Proposition 9E to the inner product in the vector space C [a, b] studied in Example 9.2.6, we obtain the Cauchy-Schwarz inequality for integrals, that for f, g ∈ C [a, b], we have b 1 /2 b f (x)g (x) dx ≤ a Chapter 9 : Real Inner Product Spaces 2 f (x) dx a 1/2 b 2 g (x) dx . a page 5 of 15 c Linear Algebra W W L Chen, 1997, 2005 Next, we investigate norm and distance. We generalize Propositions 9C and 9D. PROPOSITION 9F. Suppose that u and v are vectors in a real inner product space, and that c ∈ R. Then (a) u ≥ 0; (b) u = 0 if and only if u = 0; (c) cu = |c| u ; and (d) u + v ≤ u + v . PROPOSITION 9G. Suppose that u, v and w are vectors in a real inner product space. Then (a) d(u, v) ≥ 0; (b) d(u, v) = 0 if and only if u = v; (c) d(u, v) = d(v, u); and (d) d(u, v) ≤ d(u, w) + d(w, v). The proofs are left as exercises. The Cauchy-Schwarz inequality, as given by Proposition 9E, allows us to recover a formula of the type (3) u, v = u v cos θ. Although the number θ does not have a geometric interpretation, we can nevertheless interpret it as the angle between the two vectors u and v under the inner product , . Of particular interest is the case when cos θ = 0; in other words, when u, v = 0. Definition. Suppose that u and v are non-zero vectors in a real inner product space V . Then the unique real number θ ∈ [0, π ] satisfying (3) is called the angle between u and v with respect to the inner product , in V . Definition. Two vectors u and v in a real inner product space are said to be orthogonal if u, v = 0. Definition. Suppose that W is a subspace of a real inner product space V . A vector u ∈ V is said to be orthogonal to W if u, w = 0 for every w ∈ W . The set of all vectors u ∈ V which are orthogonal to W is called the orthogonal complement of W , and denoted by W ⊥ ; in other words, W ⊥ = {u ∈ V : u, w = 0 for every w ∈ W }. Example 9.3.3. In R3 , the non-trivial subspaces are lines and planes through the origin. Under the euclidean inner product, two non-zero vectors are orthogonal if and only if they are perpendicular. It follows that if W is a line through the origin, then W ⊥ is the plane through the origin and perpendicular to the line W . Also, if W is a plane through the origin, then W ⊥ is the line through the origin and perpendicular to the plane W . Example 9.3.4. In R4 , let us consider the two vectors u = (1, 1, 1, 0) and v = (1, 0, 1, 1). Under the euclidean inner product, we have u=v= √ 3 and u, v = 2. This verifies the Cauchy-Schwarz inequality. On the other hand, if θ ∈ [0, π ] represents the angle between u and v with respect to the euclidean inner product, then (3) holds, and we obtain cos θ = 2/3, so that θ = cos−1 (2/3). Chapter 9 : Real Inner Product Spaces page 6 of 15 c Linear Algebra W W L Chen, 1997, 2005 Example 9.3.5. In R4 , it can be shown that W = {(w1 , w2 , 0, 0) : w1 , w2 ∈ R} is a subspace. Consider now the euclidean inner product, and let A = {(0, 0, u3 , u4 ) : u3 , u4 ∈ R}. We shall show that A ⊆ W ⊥ and W ⊥ ⊆ A, so that W ⊥ = A. To show that A ⊆ W ⊥ , note that for every (0, 0, u3 , u4 ) ∈ A, we have (0, 0, u3 , u4 ), (w1 , w2 , 0, 0) = (0, 0, u3 , u4 ) · (w1 , w2 , 0, 0) = 0 for every (w1 , w2 , 0, 0) ∈ W , so that (0, 0, u3 , u4 ) ∈ W ⊥ . To show that W ⊥ ⊆ A, note that for every (u1 , u2 , u3 , u4 ) ∈ W ⊥ , we need to have (u1 , u2 , u3 , u4 ), (w1 , w2 , 0, 0) = (u1 , u2 , u3 , u4 ) · (w1 , w2 , 0, 0) = u1 w1 + u2 w2 = 0 for every (w1 , w2 , 0, 0) ∈ W . The choice (w1 , w2 , 0, 0) = (1, 0, 0, 0) requires us to have u1 = 0, while the choice (w1 , w2 , 0, 0) = (0, 1, 0, 0) requires us to have u2 = 0. Hence we must have u1 = u2 = 0, so that (u1 , u2 , u3 , u4 ) ∈ A. Example 9.3.6. Let us consider the inner product on M2,2 (R) discussed in Example 9.2.4. Let U= 1 3 0 4 and V= 4 0 2 −1 . Then U, V = 0, so that the two matrices are orthogonal. Example 9.3.7. Let us consider the inner product on P2 discussed in Example 9.2.5. Let p = p(x) = 1 + 2x + 3x2 and q = q (x) = 4 + x − 2x2 . Then p, q = 0, so that the two polynomials are orthogonal. Example 9.3.8. Let us consider the inner product on C [a, b] discussed in Example 9.2.6. In particular, let [a, b] = [0, π/2]. Suppose that f (x) = sin x − cos x and g (x) = sin x + cos x. Then π /2 f, g = π /2 0 π /2 (sin x − cos x)(sin x + cos x) dx = f (x)g (x) dx = 0 (sin2 x − cos2 x) dx = 0, 0 so that the two functions are orthogonal. Example 9.3.9. Suppose that A is an m × n matrix with real entries. Recall that if we let r1 , . . . , rm denote the vectors formed from the rows of A, then the row space of A is given by {c1 r1 + . . . + cm rm : c1 , . . . , cm ∈ R}, and is a subspace of Rn . On the other hand, the set {x ∈ Rn : Ax = 0} is called the nullspace of A, and is also a subspace of Rn . Clearly, if x belongs to the nullspace of A, then ri · x = 0 for every i = 1, . . . , m. In fact, the row space of A and the nullspace of A are orthogonal Chapter 9 : Real Inner Product Spaces page 7 of 15 c Linear Algebra W W L Chen, 1997, 2005 complements of each other under the euclidean inner product in Rn . On the other hand, the column space of A is the row space of At . It follows that the column space of A and the nullspace of At are orthogonal complements of each other under the euclidean inner product in Rm . Example 9.3.10. Suppose that u and v are orthogonal vectors in an inner product space. Then u+v 2 = u + v, u + v = u, u + 2 u, v + v, v = u 2 + v 2. This is a generalized version of Pythagoras’s theorem. Remark. We emphasize here that orthogonality depends on the choice of the inner product. Very often, a real vector space has more than one inner product. Vectors orthogonal with respect to one may not be orthogonal with respect to another. For example, the vectors u = (1, 1) and v = (1, −1) in R2 are orthogonal with respect to the euclidean inner product u, v = u1 v1 + u2 v2 , but not orthogonal with respect to the weighted euclidean inner product u, v = 2u1 v1 + u2 v2 . 9.4. Orthogonal and Orthonormal Bases Suppose that v1 , . . . , vr are vectors in a real vector space V . We often consider linear combinations of the type c1 v1 + . . . + cr vr , where c1 , . . . , cr ∈ R. The set span{v1 , . . . , vr } = {c1 v1 + . . . + cr vr : c1 , . . . , cr ∈ R} of all such linear combinations is called the span of the vectors v1 , . . . , vr . We also say that the vectors v1 , . . . , vr span V if span{v1 , . . . , vr } = V ; in other words, if every vector in V can be expressed as a linear combination of the vectors v1 , . . . , vr . It can be shown that span{v1 , . . . , vr } is a subspace of V . Suppose further that W is a subspace of V and v1 , . . . , vr ∈ W . Then span{v1 , . . . , vr } ⊆ W . On the other hand, the spanning set {v1 , . . . , vr } may contain more vectors than are necessary to describe all the vectors in the span. This leads to the idea of linear independence. Definition. Suppose that v1 , . . . , vr are vectors in a real vector space V . (LD) We say that v1 , . . . , vr are linearly dependent if there exist c1 , . . . , cr ∈ R, not all zero, such that c1 v1 + . . . + cr vr = 0. (LI) We say that v1 , . . . , vr are linearly independent if they are not linearly dependent; in other words, if the only solution of c1 v1 + . . . + cr vr = 0 in c1 , . . . , cr ∈ R is given by c1 = . . . = cr = 0. Definition. Suppose that v1 , . . . , vr are vectors in a real vector space V . We say that {v1 , . . . , vr } is a basis for V if the following two conditions are satisfied: (B1) We have span{v1 , . . . , vr } = V . (B2) The vectors v1 , . . . , vr are linearly independent. Suppose that {v1 , . . . , vr } is a basis for a real vector space V . Then it can be shown that every element u ∈ V can be expressed uniquely in the form u = c1 v1 + . . . + cr vr , where c1 , . . . , cr ∈ R. We shall restrict our discussion to finite-dimensional real vector spaces. A real vector space V is said to be finite-dimensional if it has a basis containing only finitely many elements. Suppose that {v1 , . . . , vn } Chapter 9 : Real Inner Product Spaces page 8 of 15 c Linear Algebra W W L Chen, 1997, 2005 is such a basis. Then it can be shown that any collection of more than n vectors in V must be linearly dependent. It follows that any two bases for V must have the same number of elements. This common number is known as the dimension of V . It can be shown that if V is a finite-dimensional real vector space, then any finite set of linearly independent vectors in V can be expanded, if necessary, to a basis for V . This establishes the existence of a basis for any finite-dimensional vector space. On the other hand, it can be shown that if the dimension of V is equal to n, then any set of n linearly independent vectors in V is a basis for V . Remark. The above is discussed in far greater detail, including examples and proofs, in Chapter 5. The purpose of this section is to add the extra ingredient of orthogonality to the above discussion. Definition. Suppose that V is a finite-dimensional real inner product space. A basis {v1 , . . . , vn } of V is said to be an orthogonal basis of V if vi , vj = 0 for every i, j = 1, . . . , n satisfying i = j . It is said to be an orthonormal basis if it satisfies the extra condition that vi = 1 for every i = 1, . . . , n. Example 9.4.1. The usual basis {v1 , . . . , vn } in Rn , where vi = (0, . . . , 0, 1, 0, . . . , 0) i−1 n−i for every i = 1, . . . , n, is an orthonormal basis of Rn with respect to the euclidean inner product. Example 9.4.2. The vectors v1 = (1, 1) and v2 = (1, −1) are linearly independent in R2 and satisfy v1 , v2 = v1 · v2 = 0. It follows that {v1 , v2 } is an orthogonal basis of R2 with respect to the euclidean inner product. Can you find an orthonormal basis of R2 by normalizing v1 and v2 ? It is theoretically very simple to express any vector as a linear combination of the elements of an orthogonal or orthonormal basis. PROPOSITION 9H. Suppose that V is a finite-dimensional real inner product space. If {v1 , . . . , vn } is an orthogonal basis of V , then for every vector u ∈ V , we have u= u, v1 u, vn v1 + . . . + vn . v1 2 vn 2 Furthermore, if {v1 , . . . , vn } is an orthonormal basis of V , then for every vector u ∈ V , we have u = u, v1 v1 + . . . + u, vn vn . Proof. Since {v1 , . . . , vn } is a basis of V , there exist unique c1 , . . . , cn ∈ R such that u = c1 v1 + . . . + cn vn . For every i = 1, . . . , n, we have u, vi = c1 v1 + . . . + cn vn , vi = c1 v1 , vi + . . . + cn vn , vi = ci vi , vi since vj , vi = 0 if j = i. Clearly vi = 0, so that vi , vi = 0, and so ci = u, vi vi , vi for every i = 1, . . . , n. The first assertion follows immediately. For the second assertion, note that vi , vi = 1 for every i = 1, . . . , n. Chapter 9 : Real Inner Product Spaces page 9 of 15 c Linear Algebra W W L Chen, 1997, 2005 Collections of vectors that are orthogonal to each other are very useful in the study of vector spaces, as illustrated by the following important result. PROPOSITION 9J. Suppose that the non-zero vectors v1 , . . . , vr in a finite-dimensional real inner product space are pairwise orthogonal. Then they are linearly independent. Proof. Suppose that c1 , . . . , cr ∈ R and c1 v1 + . . . + cr vr = 0. Then for every i = 1, . . . , r, we have 0 = 0, vi = c1 v1 + . . . + cr vr , vi = c1 v1 , vi + . . . + cr vr , vi = ci vi , vi since vj , vi = 0 if j = i. Clearly vi = 0, so that vi , vi = 0, and so we must have ci = 0 for every i = 1, . . . , r. It follows that c1 = . . . = cr = 0. Of course, the above is based on the assumption that an orthogonal basis exists. Our next task is to show that this is indeed the case. Our proof is based on a technique which orthogonalizes any given basis of a vector space. PROPOSITION 9K. Every finite-dimensional real inner product space has an orthogonal basis, and hence also an orthonormal basis. Remark. We shall prove Proposition 9K by using the Gram-Schmidt process. The central idea of this process, in its simplest form, can be described as follows. Suppose that v1 and u2 are two non-zero vectors in an inner product space, not necessarily orthogonal to each other. We shall attempt to remove some scalar multiple α1 v1 from u2 so that v2 = u2 − α1 v1 is orthogonal to v1 ; in other words, we wish to find a suitable real number α1 such that v1 , v2 = v1 , u2 − α1 v1 = 0. The idea is illustrated in the picture below. ;;; W ;;;; ; ;;;;; ;;;;;; ;;;;;; ;; J ;; ;; ;; ;; ;; / / o / o / o / o o / o o / o o / 2 o / o o 1 / o o / o o / o / o o / o 2/ o / o / o / o / o / o / o / o / / o o o u v ;; ;; ; 7 v We clearly need v1 , u2 − α1 v1 , v1 = 0, and α1 = v1 , u2 v1 , u2 = v 1 , v1 v1 2 and v2 = u2 − is a suitable choice, so that (4) v1 v1 , u2 v1 v1 2 are now orthogonal. Suppose in general that v1 , . . . , vs and us+1 are non-zero vectors in an inner product space, where v1 , . . . , vs are pairwise orthogonal. We shall attempt to remove some linear combination Chapter 9 : Real Inner Product Spaces page 10 of 15 c Linear Algebra W W L Chen, 1997, 2005 α1 v1 + . . . + αs vs from us+1 so that vs+1 = us+1 − α1 v1 − . . . − αs vs is orthogonal to each of v1 , . . . , vs ; in other words, we wish to find suitable real numbers α1 , . . . , αs such that vi , vs+1 = vi , us+1 − α1 v1 − . . . − αs vs = 0 for every i = 1, . . . , s. We clearly need vi , us+1 − α1 vi , v1 − . . . − αs vi , vs = vi , us+1 − αi vi , vi = 0, and αi = vi , us+1 vi , us+1 = vi , vi vi 2 is a suitable choice, so that (5) v1 , . . . , vs and vs+1 = us+1 − v1 , us+1 vs , us+1 v1 − . . . − vs 2 v1 vs 2 are now pairwise orthogonal. Example 9.4.3. The vectors u1 = (1, 2, 1, 0), u3 = (2, −10, 0, 0), u2 = (3, 3, 3, 0), u4 = (−2, 1, −6, 2) are linearly independent in R4 , since 1 2 det 1 0 3 3 3 0 2 −10 0 0 −2 1 = 0. −6 2 Hence {u1 , u2 , u3 , u4 } is a basis of R4 . Let us consider R4 as a real inner product space with the euclidean inner product, and apply the Gram-Schmidt process to this basis. We have v1 = u1 = (1, 2, 1, 0), v1 , u2 (1, 2, 1, 0), (3, 3, 3, 0) v2 = u2 − v1 = (3, 3, 3, 0) − (1, 2, 1, 0) 2 v1 (1, 2, 1, 0) 2 12 = (3, 3, 3, 0) − (1, 2, 1, 0) = (3, 3, 3, 0) + (−2, −4, −2, 0) = (1, −1, 1, 0), 6 v1 , u3 v2 , u3 v3 = u3 − v1 − v2 2 v1 v2 2 (1, 2, 1, 0), (2, −10, 0, 0) (1, −1, 1, 0), (2, −10, 0, 0) = (2, −10, 0, 0) − (1, 2, 1, 0) − (1, −1, 1, 0) (1, 2, 1, 0) 2 (1, −1, 1, 0) 2 18 12 = (2, −10, 0, 0) + (1, 2, 1, 0) − (1, −1, 1, 0) 6 3 = (2, −10, 0, 0) + (3, 6, 3, 0) + (−4, 4, −4, 0) = (1, 0, −1, 0), v1 , u4 v2 , u4 v3 , u4 v4 = u4 − v1 − v2 − v3 2 2 v1 v2 v3 2 (1, 2, 1, 0), (−2, 1, −6, 2) = (−2, 1, −6, 2) − (1, 2, 1, 0) (1, 2, 1, 0) 2 (1, −1, 1, 0), (−2, 1, −6, 2) (1, 0, −1, 0), (−2, 1, −6, 2) − (1, −1, 1, 0) − (1, 0, −1, 0) 2 (1, −1, 1, 0) (1, 0, −1, 0) 2 6 9 4 = (−2, 1, −6, 2) + (1, 2, 1, 0) + (1, −1, 1, 0) − (1, 0, −1, 0) 6 3 2 = (−2, 1, −6, 2) + (1, 2, 1, 0) + (3, −3, 3, 0) + (−2, 0, 2, 0) = (0, 0, 0, 2). Chapter 9 : Real Inner Product Spaces page 11 of 15 c Linear Algebra W W L Chen, 1997, 2005 It is easy to verify that the four vectors v1 = (1, 2, 1, 0), v2 = (1, −1, 1, 0), v3 = (1, 0, −1, 0), v4 = (0, 0, 0, 2) are pairwise orthogonal, so that {v1 , v2 , v3 , v4 } is an orthogonal basis of R4 . Normalizing each of these four vectors, we obtain the corresponding orthonormal basis 1 2 1 √ , √ , √ ,0 , 6 6 6 1 1 1 √ , −√ , √ , 0 , 3 3 3 1 1 √ , 0, − √ , 0 , (0, 0, 0, 1) . 2 2 Proof of Proposition 9K. Suppose that the vector space V has dimension of n. Then it has a basis of the type {u1 , . . . , un }. We now let v1 = u1 , and define v2 , . . . , vn inductively by (4) and (5) to obtain a set of pairwise orthogonal vectors {v1 , . . . , vn }. Clearly none of these n vectors is zero, for if vs+1 = 0, then it follows from (5) that v1 , . . . , vs , us+1 , and hence u1 , . . . , us , us+1 , are linearly dependent, clearly a contradiction. It now follows from Proposition 9J that v1 , . . . , vn are linearly independent, and so must form a basis of V . This proves the first assertion. To prove the second assertion, observe that each of the vectors v1 vn ,..., v1 vn has norm 1. Example 9.4.4. Consider the real inner product space P2 , where for polynomials p = p(x) = p0 + p1 x + p2 x2 and q = q (x) = q0 + q1 x + q2 x2 , the inner product is defined by p, q = p0 q0 + p1 q1 + p2 q2 . The polynomials u1 = 3 + 4x + 5x2 , are linearly independent in P2 , since u2 = 9 + 12x + 5x2 , 3 det 4 5 9 12 5 u3 = 1 − 7x + 25x2 1 −7 = 0. 25 Hence {u1 , u2 , u3 } is a basis of P2 . Let us apply the Gram-Schmidt process to this basis. We have v1 = u1 = 3 + 4x + 5x2 , v1 , u2 3 + 4x + 5x2 , 9 + 12x + 5x2 v1 = (9 + 12x + 5x2 ) − (3 + 4x + 5x2 ) v2 = u2 − v1 2 3 + 4x + 5x2 2 100 = (9 + 12x + 5x2 ) − (3 + 4x + 5x2 ) = (9 + 12x + 5x2 ) + (−6 − 8x − 10x2 ) = 3 + 4x − 5x2 , 50 v1 , u3 v2 , u3 v3 = u3 − v1 − v2 v1 2 v2 2 3 + 4x + 5x2 , 1 − 7x + 25x2 = (1 − 7x + 25x2 ) − (3 + 4x + 5x2 ) 3 + 4x + 5x2 2 3 + 4x − 5x2 , 1 − 7x + 25x2 − (3 + 4x − 5x2 ) 3 + 4x − 5x2 2 100 150 = (1 − 7x + 25x2 ) − (3 + 4x + 5x2 ) + (3 + 4x − 5x2 ) 50 50 = (1 − 7x + 25x2 ) + (−6 − 8x − 10x2 ) + (9 + 12x − 15x2 ) = 4 − 3x + 0x2 . Chapter 9 : Real Inner Product Spaces page 12 of 15 c Linear Algebra W W L Chen, 1997, 2005 It is easy to verify that the three polynomials v2 = 3 + 4x − 5x2 , v1 = 3 + 4x + 5x2 , v3 = 4 − 3x + 0x2 are pairwise orthogonal, so that {v1 , v2 , v3 } is an orthogonal basis of P2 . Normalizing each of these three polynomials, we obtain the corresponding orthonormal basis 3 3 43 4 5 4 5 √ + √ x + √ x2 , √ + √ x − √ x2 , − x + 0x2 . 55 50 50 50 50 50 50 9.5. Orthogonal Projections The Gram-Schmidt process is an example of using orthogonal projections. The geometric interpretation of v2 = u2 − v1 , u2 v1 v1 2 is that we have removed from u2 its orthogonal projection on v1 ; in other words, we have removed from u2 the component of u2 which is “parallel” to v1 , so that the remaining part must be “perpendicular” to v1 . It is natural to consider the following question. Suppose that V is a finite-dimensional real inner product space, and that W is a subspace of V . Given any vector u ∈ V , can we write u = w + p, where w ∈ W and p ∈ W ⊥ ? If so, is this expression unique? The following result answers these two questions in the affirmative. PROPOSITION 9L. Suppose that V is a finite-dimensional real inner product space, and that W is a subspace of V . Suppose further that {v1 , . . . , vr } is an orthogonal basis of W . Then for any vector u∈V, w= u, v1 u, vr v1 + . . . + vr v1 2 vr 2 is the unique vector satisfying w ∈ W and u − w ∈ W ⊥ . Proof. Note that the orthogonal basis {v1 , . . . , vr } of W can be extended to a basis {v1 , . . . , vr , ur+1 , . . . , un } of V which can then be orthogonalized by the Gram-Schmidt process to an orthogonal basis {v1 , . . . , vr , vr+1 , . . . , vn } of V . Clearly vr+1 , . . . , vn ∈ W ⊥ . Suppose now that u ∈ V . Then u can be expressed as a linear combination of v1 , . . . , vn in a unique way. By Proposition 9H, this unique expression is given by u= u, v1 u, vn u, vr+1 u, vn v1 + . . . + vn = w + vr+1 + . . . + vn . v1 2 vn 2 vr+1 2 vn 2 Clearly u − w ∈ W ⊥ . Chapter 9 : Real Inner Product Spaces page 13 of 15 c Linear Algebra W W L Chen, 1997, 2005 Definition. The vector w in Proposition 9L is called the orthogonal projection of u on the subspace W , and denoted by projW u. The vector p = u − w is called the component of u orthogonal to the subspace W . Example 9.5.1. Recall Example 9.4.3. Consider the subspace W = span{u1 , u2 }. Note that v1 and v2 can each be expressed as a linear combination of u1 and u2 , and that u1 and u2 can each be expressed as a linear combination of v1 and v2 . It follows that {v1 , v2 } is an orthogonal basis of W . This basis can be extended to an orthogonal basis {v1 , v2 , v3 , v4 } of R4 . It follows that W ⊥ = span{v3 , v4 }. Problems for Chapter 9 1. In each of the following, determine whether , checking whether conditions (IP1)–(IP4) hold: a) R2 ; u, v = 2u1 v1 − u2 v2 2 2 2 c) R3 ; u, v = u2 v1 + u2 v2 + u2 v3 1 2 3 is an inner product in the given vector space by 2. Consider the vector space R2 . Suppose that is the inner product generated by the matrix , A= Evaluate each of the following: a) (1, 2), (2, 3) b) R2 ; u, v = u1 v1 + 2u1 v2 + u2 v2 2 2 1 3 . b) (1, 2) c) d((1, 2), (2, 3)) 3. Suppose that the vectors u, v, w in an inner product space V satisfy u, v = 2, v, w = −3, u, w = 5, u = 1, v = 2 and w = 7. Evaluate each of the following: a) u + v, v + w b) 2v − w, 3u + 2w c) u − v − 2w, 4u + v d) u + v e) 2w − v f ) u − 2v + 4w 4. Suppose that u and v are two non-zero vectors in the real vector space R2 . Follow the steps below to establish the existence of a real inner product , on R2 such that u, v = 0. a) Explain, in terms of the euclidean inner product, why we may restrict our discussion to vectors of the form u = (x, y ) and v = (ky, −kx), where x, y, k ∈ R satisfy (x, y ) = (0, 0) and k = 0. b) Explain next why we may further restrict our discussion to vectors of the form u = (x, y ) and v = (y, −x), where x, y ∈ R satisfy (x, y ) = (0, 0). c) Let u = (x, y ) and v = (y, −x), where x, y ∈ R and (x, y ) = (0, 0). Consider the inner product on R2 generated by the real matrix A= ab bc , where ac = b2 . Show that u, v = (a2 − c2 )xy + b(a + c)(y 2 − x2 ). d) Suppose that x2 = y 2 . Show that the choice a > c > b = 0 will imply u, v = 0. e) Suppose that x2 = y 2 . Show that the choice c = a > b > 0 will imply u, v = 0. 5. Consider the real vector space R2 . a) Find two distinct non-zero vectors u, v ∈ R2 such that u, v = 0 for every weighted euclidean inner product on R2 . b) Find two distinct non-zero vectors u, v ∈ R2 such that u, v = 0 for any inner product on R2 . Chapter 9 : Real Inner Product Spaces page 14 of 15 c Linear Algebra W W L Chen, 1997, 2005 6. For each of the following inner product spaces and subspaces W , find W ⊥ : a) R2 (euclidean inner product); W = {(x, y ) ∈ R2 : x + 2y = 0}. b) M2,2 (R) (inner product discussed in Section 9.2); ta 0 0 tb W= :t∈R , where a and b are non-zero. 7. Suppose that {v1 , . . . , vn } is a basis for a real inner product space V . Does there exist v ∈ V which is orthogonal to every vector in this basis? 8. Use the Cauchy-Schwarz inequality to prove that (a cos θ + b sin θ)2 ≤ a2 + b2 for every a, b, θ ∈ R. [Hint: First find a suitable real inner product space.] 9. Prove Proposition 9F. 10. Show that u, v = 1 4 u+v 2 − 1 4 u−v 2 for any u and v in a real inner product space. 11. Suppose that {v1 , . . . , vn } is an orthonormal basis of a real inner product space V . Show that for every u ∈ V , we have u 2 = u, v1 2 + . . . + u, vn 2 . 12. Show that if v1 , . . . , vn are pairwise orthogonal in a real inner product space V , then v1 + . . . + vn 2 = v1 2 + . . . + vn 2 . 13. Show that v1 = (2, −2, 1), v2 = (2, 1, −2) and v3 = (1, 2, 2) form an orthogonal basis of R3 under the euclidean inner product. Then write u = (−1, 0, 2) as a linear combination of v1 , v2 , v3 . 14. Let u1 = (2, 2, −1), u2 = (4, 1, 1) and u3 = (1, 10, −5). Show that {u1 , u2 , u3 } is a basis of R3 , and apply the Gram-Schmidt process to this basis to find an orthonormal basis of R3 . 15. Show that the vectors u1 = (0, 2, 1, 0), u2 = (1, −1, 0, 0), u3 = (1, 2, 0, −1) and u4 = (1, 0, 0, 1) form a basis of R4 . Then apply the Gram-Schmidt process to find an orthogonal basis of R4 . Find also the corresponding orthonormal basis of R4 . 16. Consider the vector space P2 with the inner product 1 p, q = p(x)q (x) dx. 0 Apply the Gram-Schmidt process to the basis {1, x, x2 } to find an orthogonal basis of P2 . Find also the corresponding orthonormal basis of P2 . 17. Suppose that we apply the Gram-Schmidt process to non-zero vectors u1 , . . . , un without first checking that these form a basis of the inner product space, and obtain vs = 0 for some s = 1, . . . , n. What conclusion can we draw concerning the collection {u1 , . . . , un }? Chapter 9 : Real Inner Product Spaces page 15 of 15 ...
View Full Document

This note was uploaded on 06/13/2009 for the course TAM 455 taught by Professor Petrina during the Fall '08 term at Cornell University (Engineering School).

Ask a homework question - tutors are online