lecture_16-18 - 67 Lecture 16-18 In many practical...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 67 Lecture 16-18 In many practical applications, we are asked to approximate a given signal using a linear combination of a fixed collection of elementary signals. Recall that while studying DT signals, in the first part of the course, we looked at at least 3 ways of representing arbitrary DT signals using elementary signals as building blocks. In those cases we required exact representations. However, if there is some constraint on the available resources, in the sense that the collection of elementary signals is not enough for exact representations, then how to obtain the best representation of a given signal ? This question is answered by projection theorem . Projection Theorem: Let S be a Hilbert space, and let V be a subspace of S . For any vector (arbitrary) x ∈ S , 1. There exist a unique solution to the optimization problem arg min v ∈ V bardbl x − v bardbl where bardbl · bardbl denotes the induced norm. 2. v is the minimizer of the above optimization problem if and only if ( x − v ) ⊥ v, for all v ∈ V Intuition from 3-dimensional euclidean space: Let S be a 3D euclidean space. Let V = span { (1,0,0), (0,1,0) } . See Fig. 58. Let ( x,y ) = x 1 y 1 + x 2 y 2 + x 3 y 3 bardbl x − y bardbl = radicalbig ( x 1 − y 1 ) 2 + ( x 2 − y 2 ) 2 + ( x 3 − y 3 ) 2 In this case we can look at the optimization problem as: arg min ( v 1 ,v 2 ,v 3 ) ∈ V radicalbig ( x 1 − v 1 ) 2 + ( x 2 − v 2 ) 2 + ( x 3 − v 3 ) 2 We can solve this by inspection and obtain the optimizer as V = ( x 1 ,x 2 , 0) . Observe that in this case the error vector is given by (0 , ,x 3 ) and it is orthogonal to every vector in V. Projection theorem says that this picture can be extended to arbitrary Hilbert space. Proof: We will look at the proof of Statement (b). The proof of Statement (a) is technical in nature and is out of the scope of this course. Necessary Condition: If V is the minimizer then ( x − v ) ⊥ V for all v ∈ V Assume: V is the minimizer and there exist a vector v ′ ∈ V such that ( x − v ) is not ⊥ V . Let ( ( x − v ) ,v ′ ) = δ negationslash = 0 . z = v + δ bardbl v ′ bardbl 2 v ′ . 68 x v e Figure 58: Projection Theorem Since v ∈ V and v ′ ∈ V , we have z ∈ V . ⇒ bardbl x − z bardbl 2 = bardbl x − v − δ bardbl v ′ bardbl 2 v ′ bardbl 2 = bardbl x − v bardbl 2 + | δ | 2 bardbl v ′ bardbl 4 bardbl v ′ bardbl 2 + ( x − v , − δv ′ bardbl v bardbl 2 ) + ( − δv ′ bardbl v bardbl 2 ,x − v ) = bardbl x − v bardbl 2 + | δ | 2 bardbl v bardbl 4 + − ( δ ) ∗ δ bardbl v ′ bardbl 2 + − ( δ ) δ ∗ bardbl v ′ bardbl 2 = bardbl x − v bardbl 2 − | δ | 2 bardbl v bardbl 2 ⇒ bardbl x − z bardbl 2 < bardbl x − v bardbl 2 ⇒ V cannot be the minimizer....
View Full Document

This note was uploaded on 01/10/2012 for the course EECS eecs551 taught by Professor J during the Spring '11 term at University of Michigan.

Page1 / 12

lecture_16-18 - 67 Lecture 16-18 In many practical...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online