can be rewritten as k β Σ α k 2 2 δ k α k 2 2 R X k 1 β k σ k α k 2 δα k 2 8 We

# Can be rewritten as k β σ α k 2 2 δ k α k 2 2 r

• Notes
• 18

This preview shows page 11 - 15 out of 18 pages.

can be rewritten as k β - Σ α k 2 2 + δ k α k 2 2 = R X k =1 ( β [ k ] - σ k α [ k ]) 2 + δα [ k ] 2 . (8) We can minimize this sum simply by minimizing each term indepen- dently. Since d [ k ] ( β [ k ] - σ k α [ k ]) 2 + δα [ k ] 2 = - 2 β [ k ] σ k + 2 σ 2 k α [ k ] + 2 δα [ k ] , we need α [ k ] = σ k σ 2 k + δ β [ k ] . Putting this back in vector form, ( 8 ) is minimized by ˆ α tik = ( Σ 2 + δ I ) - 1 Σ β , and so the minimizer to ( 6 ) is ˆ x tik = V ˆ α tik = V ( Σ 2 + δ I ) - 1 Σ U T y . (9) We can get a better feel for what Tikhonov regularization is doing by comparing it directly to the pseudo-inverse. The least-squares 59 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 21:13, November 3, 2019 Subscribe to view the full document.

reconstruction ˆ x ls can be written as ˆ x ls = V Σ - 1 U T y = R X r =1 1 σ r h y , u r i v r , while the Tikhonov reconstruction ˆ x tik derived above is ˆ x tik = R X r =1 σ r σ 2 r + δ h y , u r i v r . (10) Notice that when σ r is much larger than δ , σ r σ 2 r + δ 1 σ r , σ r δ, but when σ r is small σ r σ 2 r + δ 0 , σ r δ. Thus the Tikhonov reconstruction modifies the important parts (com- ponents where the σ r are large) of the pseudo-inverse very little, while ensuring that the unimportant parts (components where the σ r are small) affect the solution only by a very small amount. This damp- ing of the singular values, is illustrated below. σ r σ 2 r + δ 0 0.5 1 1.5 2 2.5 3 0 1 2 3 4 5 6 σ r 60 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 21:13, November 3, 2019 Above, we see the damped multipliers σ r / ( σ 2 r + δ ) versus σ r for δ = 0 . 1 (blue), δ = 0 . 05 (red), and δ = 0 . 01 (green). The black dotted line is 1 r , the least-squares multiplier. Notice that for large σ r ( σ r > 2 δ , say), the damping has almost no effect. This damping makes the Tikhonov reconstruction exceptionally sta- ble; large multipliers never appear in the reconstruction ( 10 ). In fact it is easy to check that σ r σ 2 r + δ 1 2 δ no matter the value of σ r . Tikhonov Error Analysis Given noisy observations y = Ax 0 + e , how well will Tikhonov reg- ularization work? The answer to this questions depends on multiple factors including the choice of δ , the nature of the perturbation e , and how well x 0 can be approximated using a linear combination of the singular vectors v r corresponding to the large (relative to δ ) singular values. Since a closed-form expression for the solution to ( 6 ) exists, we can quantify these trade-offs precisely. We compare the Tikhonov reconstruction to the reconstruction we would obtain if we used standard least-squares on perfectly noise- free observations y clean = Ax 0 . This noise-free reconstruction can 61 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 21:13, November 3, 2019 Subscribe to view the full document.

be written as x pinv = A y clean = A Ax 0 = V Σ - 1 U T U Σ V T x 0 = V V T x 0 = R X r =1 h x 0 , v r i v r . The vector x pinv is the orthogonal projection of x 0 onto the row space (everything orthogonal to the null space) of A . If A has full column rank, then x pinv = x 0 . If not, then the application of A destroys the part of x 0 that is not in x pinv , and so we only attempt to recover the “visible” components. In some sense, x pinv contains all of the components of x 0 that we could ever hope to recover, and has them preserved perfectly.  • Fall '08
• Staff

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern