These types of systems are prevalent in array signal processing and machine

# These types of systems are prevalent in array signal

• Notes
• 31

This preview shows page 15 - 19 out of 31 pages.

These types of systems are prevalent in array signal processing and machine learning. We will see that if R N , this system can be solved in (much) faster than O ( N 3 ) time. Note that while BB T is not at all invertible (since it is rank defi- cient), γ I + BB T will be. To see this, set z = B T x , z R R . 14 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 13:22, November 13, 2019 Subscribe to view the full document.

Then we can solve the system by jointly solving for x and z : γ x + Bz = b (5) B T x - z = 0 . (6) Solving the first equations ( 5 ) yields x = γ - 1 ( b - Bz ) , and then plugging this into ( 6 ) gives us γ - 1 B T ( b - Bz ) - z = 0 ( γ I + B T B ) z = B T b and so z = ( γ I + B T B ) - 1 B T b . But notice that this is an R × R system of equations. So it takes O ( NR 2 ) to construct γ I + B T B , then O ( R 3 ) to solve for z , then O ( NR ) to calculate Bz (and hence find x ). The dominant cost in all of this is O ( NR 2 ), which is much less than O ( N 3 ) if R N . Circulant systems . A circulant matrix has the form H = h 0 h N - 1 h N - 2 · · · h 1 h 1 h 0 h N - 1 · · · h 2 h 2 h 1 h 0 . . . . . . . . . . . . h N - 1 h N - 1 h 1 h 0 15 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 13:22, November 13, 2019 For H symmetric, we have h k = h N - k for k = 1 , . . . , N - 1, although symmetry does not play too big a role in exploiting this structure. Circulant matrices have two very nice properties: We know their eigenvectors already — they are the discrete harmonic sinusoids (i.e. the columns of the N × N DFT ma- trix). Transforming into the eigenbasis is fast thanks to the FFT (which is O ( N log N )). We can write H = F Λ F H , F [ m, n ] = 1 N e j 2 πmn/N and H - 1 = F Λ - 1 F H , so H - 1 b = F |{z} FFT, O ( N log N ); Λ - 1 |{z} diagonal weighting, O ( N ); F H b | {z } FFT, O ( N log N ) solving an N × N system of equations can be done in O ( N log N ) time! This is fast compared to O ( N 3 ) — on my computer, I can solve a system like this in N = 20 000 in 800 μ s (compare to 24 seconds for the general case). 16 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 13:22, November 13, 2019 Subscribe to view the full document.

Toeplitz systems . Toeplitz matrices, which are matrices that are constant along their diagonals, arise in many different signal processing applications, as they are fundamental in describing the action of linear time-invariant systems. For example, suppose we observe the discrete convolution of an unknown signal x of length N and a known sequence 2 a 0 , . . . , a L - 1 of length L . We can write the corresponding matrix equation as a 0 0 · · · 0 a 1 a 0 0 · · · 0 a 2 a 1 a 0 · · · 0 . . . a N - 1 a N - 2 · · · a 0 . . . a L - 1 a L - 2 · · · a L - N 0 a L - 1 · · · a L - N +1 . . . 0 0 · · · · · · a L - 1 x  x  . . . x [ N - 1] = y  y  y  .  • Fall '08
• Staff

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern