This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: 67 Lecture 1618 In many practical applications, we are asked to approximate a given signal using a linear combination of a fixed collection of elementary signals. Recall that while studying DT signals, in the first part of the course, we looked at at least 3 ways of representing arbitrary DT signals using elementary signals as building blocks. In those cases we required exact representations. However, if there is some constraint on the available resources, in the sense that the collection of elementary signals is not enough for exact representations, then how to obtain the best representation of a given signal ? This question is answered by projection theorem . Projection Theorem: Let S be a Hilbert space, and let V be a subspace of S . For any vector (arbitrary) x ∈ S , 1. There exist a unique solution to the optimization problem arg min v ∈ V bardbl x − v bardbl where bardbl · bardbl denotes the induced norm. 2. v is the minimizer of the above optimization problem if and only if ( x − v ) ⊥ v, for all v ∈ V Intuition from 3dimensional euclidean space: Let S be a 3D euclidean space. Let V = span { (1,0,0), (0,1,0) } . See Fig. 58. Let ( x,y ) = x 1 y 1 + x 2 y 2 + x 3 y 3 bardbl x − y bardbl = radicalbig ( x 1 − y 1 ) 2 + ( x 2 − y 2 ) 2 + ( x 3 − y 3 ) 2 In this case we can look at the optimization problem as: arg min ( v 1 ,v 2 ,v 3 ) ∈ V radicalbig ( x 1 − v 1 ) 2 + ( x 2 − v 2 ) 2 + ( x 3 − v 3 ) 2 We can solve this by inspection and obtain the optimizer as V = ( x 1 ,x 2 , 0) . Observe that in this case the error vector is given by (0 , ,x 3 ) and it is orthogonal to every vector in V. Projection theorem says that this picture can be extended to arbitrary Hilbert space. Proof: We will look at the proof of Statement (b). The proof of Statement (a) is technical in nature and is out of the scope of this course. Necessary Condition: If V is the minimizer then ( x − v ) ⊥ V for all v ∈ V Assume: V is the minimizer and there exist a vector v ′ ∈ V such that ( x − v ) is not ⊥ V . Let ( ( x − v ) ,v ′ ) = δ negationslash = 0 . z = v + δ bardbl v ′ bardbl 2 v ′ . 68 x v e Figure 58: Projection Theorem Since v ∈ V and v ′ ∈ V , we have z ∈ V . ⇒ bardbl x − z bardbl 2 = bardbl x − v − δ bardbl v ′ bardbl 2 v ′ bardbl 2 = bardbl x − v bardbl 2 +  δ  2 bardbl v ′ bardbl 4 bardbl v ′ bardbl 2 + ( x − v , − δv ′ bardbl v bardbl 2 ) + ( − δv ′ bardbl v bardbl 2 ,x − v ) = bardbl x − v bardbl 2 +  δ  2 bardbl v bardbl 4 + − ( δ ) ∗ δ bardbl v ′ bardbl 2 + − ( δ ) δ ∗ bardbl v ′ bardbl 2 = bardbl x − v bardbl 2 −  δ  2 bardbl v bardbl 2 ⇒ bardbl x − z bardbl 2 < bardbl x − v bardbl 2 ⇒ V cannot be the minimizer....
View
Full
Document
This note was uploaded on 01/10/2012 for the course EECS eecs551 taught by Professor J during the Spring '11 term at University of Michigan.
 Spring '11
 J

Click to edit the document details