can be rewritten as k β Σ α k 2 2 δ k α k 2 2 R X k 1 β k σ k α k 2 δα k 2 8 We

Can be rewritten as k β σ α k 2 2 δ k α k 2 2 r

This preview shows page 11 - 15 out of 18 pages.

can be rewritten as k β - Σ α k 2 2 + δ k α k 2 2 = R X k =1 ( β [ k ] - σ k α [ k ]) 2 + δα [ k ] 2 . (8) We can minimize this sum simply by minimizing each term indepen- dently. Since d [ k ] ( β [ k ] - σ k α [ k ]) 2 + δα [ k ] 2 = - 2 β [ k ] σ k + 2 σ 2 k α [ k ] + 2 δα [ k ] , we need α [ k ] = σ k σ 2 k + δ β [ k ] . Putting this back in vector form, ( 8 ) is minimized by ˆ α tik = ( Σ 2 + δ I ) - 1 Σ β , and so the minimizer to ( 6 ) is ˆ x tik = V ˆ α tik = V ( Σ 2 + δ I ) - 1 Σ U T y . (9) We can get a better feel for what Tikhonov regularization is doing by comparing it directly to the pseudo-inverse. The least-squares 59 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 21:13, November 3, 2019
Image of page 11

Subscribe to view the full document.

reconstruction ˆ x ls can be written as ˆ x ls = V Σ - 1 U T y = R X r =1 1 σ r h y , u r i v r , while the Tikhonov reconstruction ˆ x tik derived above is ˆ x tik = R X r =1 σ r σ 2 r + δ h y , u r i v r . (10) Notice that when σ r is much larger than δ , σ r σ 2 r + δ 1 σ r , σ r δ, but when σ r is small σ r σ 2 r + δ 0 , σ r δ. Thus the Tikhonov reconstruction modifies the important parts (com- ponents where the σ r are large) of the pseudo-inverse very little, while ensuring that the unimportant parts (components where the σ r are small) affect the solution only by a very small amount. This damp- ing of the singular values, is illustrated below. σ r σ 2 r + δ 0 0.5 1 1.5 2 2.5 3 0 1 2 3 4 5 6 σ r 60 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 21:13, November 3, 2019
Image of page 12
Above, we see the damped multipliers σ r / ( σ 2 r + δ ) versus σ r for δ = 0 . 1 (blue), δ = 0 . 05 (red), and δ = 0 . 01 (green). The black dotted line is 1 r , the least-squares multiplier. Notice that for large σ r ( σ r > 2 δ , say), the damping has almost no effect. This damping makes the Tikhonov reconstruction exceptionally sta- ble; large multipliers never appear in the reconstruction ( 10 ). In fact it is easy to check that σ r σ 2 r + δ 1 2 δ no matter the value of σ r . Tikhonov Error Analysis Given noisy observations y = Ax 0 + e , how well will Tikhonov reg- ularization work? The answer to this questions depends on multiple factors including the choice of δ , the nature of the perturbation e , and how well x 0 can be approximated using a linear combination of the singular vectors v r corresponding to the large (relative to δ ) singular values. Since a closed-form expression for the solution to ( 6 ) exists, we can quantify these trade-offs precisely. We compare the Tikhonov reconstruction to the reconstruction we would obtain if we used standard least-squares on perfectly noise- free observations y clean = Ax 0 . This noise-free reconstruction can 61 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 21:13, November 3, 2019
Image of page 13

Subscribe to view the full document.

be written as x pinv = A y clean = A Ax 0 = V Σ - 1 U T U Σ V T x 0 = V V T x 0 = R X r =1 h x 0 , v r i v r . The vector x pinv is the orthogonal projection of x 0 onto the row space (everything orthogonal to the null space) of A . If A has full column rank, then x pinv = x 0 . If not, then the application of A destroys the part of x 0 that is not in x pinv , and so we only attempt to recover the “visible” components. In some sense, x pinv contains all of the components of x 0 that we could ever hope to recover, and has them preserved perfectly.
Image of page 14
Image of page 15
  • Fall '08
  • Staff

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Ask Expert Tutors You can ask You can ask ( soon) You can ask (will expire )
Answers in as fast as 15 minutes