These types of systems are prevalent in array signal processing and machine

These types of systems are prevalent in array signal

This preview shows page 15 - 19 out of 31 pages.

These types of systems are prevalent in array signal processing and machine learning. We will see that if R N , this system can be solved in (much) faster than O ( N 3 ) time. Note that while BB T is not at all invertible (since it is rank defi- cient), γ I + BB T will be. To see this, set z = B T x , z R R . 14 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 13:22, November 13, 2019
Image of page 15

Subscribe to view the full document.

Then we can solve the system by jointly solving for x and z : γ x + Bz = b (5) B T x - z = 0 . (6) Solving the first equations ( 5 ) yields x = γ - 1 ( b - Bz ) , and then plugging this into ( 6 ) gives us γ - 1 B T ( b - Bz ) - z = 0 ( γ I + B T B ) z = B T b and so z = ( γ I + B T B ) - 1 B T b . But notice that this is an R × R system of equations. So it takes O ( NR 2 ) to construct γ I + B T B , then O ( R 3 ) to solve for z , then O ( NR ) to calculate Bz (and hence find x ). The dominant cost in all of this is O ( NR 2 ), which is much less than O ( N 3 ) if R N . Circulant systems . A circulant matrix has the form H = h 0 h N - 1 h N - 2 · · · h 1 h 1 h 0 h N - 1 · · · h 2 h 2 h 1 h 0 . . . . . . . . . . . . h N - 1 h N - 1 h 1 h 0 15 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 13:22, November 13, 2019
Image of page 16
For H symmetric, we have h k = h N - k for k = 1 , . . . , N - 1, although symmetry does not play too big a role in exploiting this structure. Circulant matrices have two very nice properties: We know their eigenvectors already — they are the discrete harmonic sinusoids (i.e. the columns of the N × N DFT ma- trix). Transforming into the eigenbasis is fast thanks to the FFT (which is O ( N log N )). We can write H = F Λ F H , F [ m, n ] = 1 N e j 2 πmn/N and H - 1 = F Λ - 1 F H , so H - 1 b = F |{z} FFT, O ( N log N ); Λ - 1 |{z} diagonal weighting, O ( N ); F H b | {z } FFT, O ( N log N ) solving an N × N system of equations can be done in O ( N log N ) time! This is fast compared to O ( N 3 ) — on my computer, I can solve a system like this in N = 20 000 in 800 μ s (compare to 24 seconds for the general case). 16 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 13:22, November 13, 2019
Image of page 17

Subscribe to view the full document.

Toeplitz systems . Toeplitz matrices, which are matrices that are constant along their diagonals, arise in many different signal processing applications, as they are fundamental in describing the action of linear time-invariant systems. For example, suppose we observe the discrete convolution of an unknown signal x of length N and a known sequence 2 a 0 , . . . , a L - 1 of length L . We can write the corresponding matrix equation as a 0 0 · · · 0 a 1 a 0 0 · · · 0 a 2 a 1 a 0 · · · 0 . . . a N - 1 a N - 2 · · · a 0 . . . a L - 1 a L - 2 · · · a L - N 0 a L - 1 · · · a L - N +1 . . . 0 0 · · · · · · a L - 1 x [0] x [1] . . . x [ N - 1] = y [0] y [1] y [2] .
Image of page 18
Image of page 19
  • Fall '08
  • Staff

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Ask Expert Tutors You can ask You can ask ( soon) You can ask (will expire )
Answers in as fast as 15 minutes