{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

ECE210a_lecture4_small

# ECE210a_lecture4_small - Matrix rank Matrix products A E...

This preview shows pages 1–7. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Matrix rank Matrix products: A E Rm”, B E 72””, AB 6 Rm” rank(AB) = rank(B) — dim(N (A) n R (B)). rank(B) +ra.nk(A)—n S rank(AB) S min{ra.nk(A),ra.nk(B)} dim=r dim=r dim=s A A BR(AT) B B 37am) /— MET) A . B B N(B) dim=m—r dim=n—r dim=p—s Roy Smith, ECE 210a, 4: 1 Matrix rank M ATA and AAT. Claim: rank(ATA) = rank(A) = rank(AT) Proof: 1: ATA) A) — d N(AT)chA) = A) %,—/ A A R(ATA) = R(AT) \Ay 72(AA1) = ’R,(A) >< ATA = A AT A M ) NH 2 m in J H 2 :1; J Roy Smith, ECE 210a, 4: 2 Normal equations Solving: Ax = b The equation, Ax = b is consistent if there exists at least one solution. Consistent case Clearly, A3: = b is consistent if and only if b E R (A). Choose a particular solution, Ant 2 b. The general solution is the set, 20,011, = {a%+ati I Aci=b, ati EN(A)} For a: 6 9165011,, Ax = A(§:+a:t) = A:1‘:+AavL = b. Normal equations: ATAa: = ATb Note that these have the same set of general solutions, because N(ATA) = N(A) => xi EN(ATA). So, for a: E \$90111, ATAa: = ATA(:E + mi) = ATAa‘: + ATAar:L = Alb. Roy Smith, ECE 210a, 4: 3 Normal equations Solving: Az = b To get a unique solution we need, b e 7?, (A) and N (A) = 0. In this case, 92 = (1434)"1 AT b. Note that ATA 6 RM" and rank(ATA) = rank(A) = n if N (A) = 0. So the inverse (ATA)71 exists. Inconsistent case: b gé R(A) But for the normal equations, ATAm = ATb, ATb e R (AT) = R (ATA) so the normal equations are always consistent. Furthermore, if N (A) = 0, then, a: = (AT/1fl ATb, uniquely solves the normal equations. How does this relate to the original equation: Ax = b? Roy Smith, ECE 210a, 4: 4 Least squares Least squares: The “best” solution to A9: = b. Deﬁne 6(.’E) 2: Am — b (residual). If Ax = b is consistent then 23501,. gives 423501“) = 0. But in general we want to minimize some measure of 6. Sum of squares: 2 672(1)) = e(z)Te(w) 2 (AT — b)T(A:z — b) = TTATAT — 2TTATb + bTb 2: f(z) i:l Objective: min f (:11) (note that f (:13) is quadratic) Calculus approach: Differentiate w.r.t. a: and set equal to zero. 6f(w) am) a?“ _ 0 aw am) 01),, Roy Smith, ECE 210a, 4: 5 Least squares Least squares aﬂx) _ axT T T T 836 317T T 333g — 823A Acc+x A Aami—2axiA b Using the fact that, 393 _ - 8f(x) _ T T T T 6—93 — 6, gives, 8567' — 2e,A Ax 26,,A b. Stacking the 6 f (:17) / 6% into a vector gives, am) = 2ATAx — 2ATb 6m ‘ Now equating this to zero implies that every minimizing solution satisﬁes, ATAx = ATb, «— Normal equations and the minimum achieved is: min f (x) = f (Tom) 2 min (xTATAx — ZTTATb + bTb) = bTb — xTATb. Roy Smith, ECE 210a, 4: 6 Least squares Converse: All solutions of the normal equations minimize the LS error. Say \$50111 is a solution of the normal equations, ATAacmln = ATb. Now consider any other at E ’R” and deﬁne: 61., = w — xsoln- Then, ft”) = (61‘ + \$90111)TATA(6.L‘ + 1750111) — 2(61: + \$50111)TATb + bTb 63.14346,r + 6fATAwmln + xT ATA6m + xT ATAxmln — zafATb — ZacmlnATb + 1% solu soln = f(ms(,1n) + 61TATA6J; (so 9350111 minimizes f ((10)) W Z 0 We can also see that 6fATA6J, = 0 implies that 613 E N (A) = N (ATA) which in this case means that 6:1. and x are also solutions of the the normal equations. Conclusion: z is a LS solution 4:) z solves the normal equations. Caveat Emptor: For theoretical use only! ATAx = ATb is much more likely to have numerical problems than A2: = b. Roy Smith, ECE 210a, 4: 7 Least squares Least squares example: linear (aﬁine) data ﬁtting b b = + t Data: (t1, ()1) 01 5 (752,52) (tin: bm) t1 t Objective: Find a and ﬂ so that t3 t4 t5 ’56 b = oz + ﬁt approximates the data. 1 t1 171 Linear ﬁtting equations: 1 t 5 = b 771, \ ( 777, W {I} V A b Roy Smith, ECE 210a, 4: 8 Least squares Least squares: Fitting polynomials to data Polynomial form: b = a0 + alt + agt2 + - - - + a,,,1t”’1 The matrix formulation is now: 1 t1 t? tlfil 010 I71 1 t2 t3 télil Oil b2 1 13,, t3, t;;,f1 01",, b,,, —,—1 W V A a; b In this case A is a Vandermonde matrix (see 2.8). If n g m (the number of data points is greater than or equal to the order of the polynomial) and t,- 7é t,- ifi aéj then N(A) = 0. Therefore the LS solution is unique and (theoretically) given by, \$0,, = (AT/0’1 ATb. Roy Smith, ECE 210a, 4: 9 Similarity and bases Coordinate matrix representations (see 2.16) Given two sets of basis vectors of the same dimension, BX = {x1,...,:c,,} and By = {y1,...,y,,}, Where X = span(BX) and 3} = span(By). The coordinate matrix representation of an operator: T : X —> 3} is, 15lele = [lT(\$1)lBy [T(m2)l5y [T(\$7I)lb’y] In the case of an identity operator, [T(x,-)] By = [(1);]By. We can deﬁne the operator, P, P I: By [1131” The matrix, P, effectively changes the coordinates of a vector in the basis BX to its coordinates in the basis By. [vi/5y : Pllea” Roy Smith, ECE 210a, 4: 10 Similarity and bases Change of basis Note that P is non-singular and unique. P’1 maps a vector in the basis By to its coordinates in the basis BX. Matrix operators Say we have an operator, A, deﬁned in terms of the By basis, [UlBy = [AlBy [wlBy- To express this in another basis, apply the change of basis matrix to each of the vectors, 1m = P 1111 = 1A1 P1w1 so, llex = P71 [Alla-P [“1le- and we can identify this as the operator in the basis By. [Ale = P71[A]ByP. Roy Smith, ECE 210a, 4: 11 Similarity and bases Similarity transforms Matrices A and B are similar if there exists an invertible matrix Q such that, A = Q’lBQ. The change of basis operation is a similarity transform on the operator’s coordinate matrix. Invariance under similarity The following properties (amongst others) are preserved by the similarity transform, A = 624362. — rank(A) = rank(B) — Eigenvalues(A) = eigenvalues(B) — trace (A) = trace (B) The trace of a matrix is deﬁned as, trace (A) = Z (1,, (sum of the diagonal elements) {:1 Roy Smith, ECE 210a, 4: 12 Example: Discrete Fourier Transform (DFT) Usual interpretation: sampling a time-domain signal: wk 2 x(t)|t:krr, k = 0, . . . , N — 1, (T is the sample period) The DFT is deﬁned as: N—l 7.1271» X(i) = Zx(k)e , i=0,...,N—1, k:0 and the inverse DFT is given by: Nil 111(k) = %ZX(i)eJ—, k=0,...,N—1. i:[) 72W Simplifying notation: 5 := e V Ni] X(z') = 295005“, i=0,...,N—1, k:0 1N71 w(k) = NZXW’“: k=0,...,N—1. i:(J DFT Express this as a matrix operation: X(0) 1 1 1 1 X0) 1 € £2 6H X<2> = 1 52 54 5H X(N—1) 1 éiV:Hl 8TH? E —,_/ EN The Fourier matrix, FN, has the properties: _ EV E CINXN; — Ev is a Vandermonde matrix (see 2.8); — PEI exists. — others to be discussed later... \$(0) 03(1) 03(2) z(N:— 1) Similarity and bases Roy Smith, ECE 210a, 4: 13 Similarity and bases It can be viewed as a change of basis between the time-domain representation, and a discrete-frequency representation. Roy Smith, ECE 210a, 4: 14 ...
View Full Document

{[ snackBarMessage ]}