{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

ECE210a_lecture4_small

ECE210a_lecture4_small - Matrix rank Matrix products A E...

Info iconThis preview shows pages 1–7. Sign up to view the full content.

View Full Document Right Arrow Icon
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Background image of page 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Background image of page 4
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Background image of page 6
Background image of page 7
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Matrix rank Matrix products: A E Rm”, B E 72””, AB 6 Rm” rank(AB) = rank(B) — dim(N (A) n R (B)). rank(B) +ra.nk(A)—n S rank(AB) S min{ra.nk(A),ra.nk(B)} dim=r dim=r dim=s A A BR(AT) B B 37am) /— MET) A . B B N(B) dim=m—r dim=n—r dim=p—s Roy Smith, ECE 210a, 4: 1 Matrix rank M ATA and AAT. Claim: rank(ATA) = rank(A) = rank(AT) Proof: 1: ATA) A) — d N(AT)chA) = A) %,—/ A A R(ATA) = R(AT) \Ay 72(AA1) = ’R,(A) >< ATA = A AT A M ) NH 2 m in J H 2 :1; J Roy Smith, ECE 210a, 4: 2 Normal equations Solving: Ax = b The equation, Ax = b is consistent if there exists at least one solution. Consistent case Clearly, A3: = b is consistent if and only if b E R (A). Choose a particular solution, Ant 2 b. The general solution is the set, 20,011, = {a%+ati I Aci=b, ati EN(A)} For a: 6 9165011,, Ax = A(§:+a:t) = A:1‘:+AavL = b. Normal equations: ATAa: = ATb Note that these have the same set of general solutions, because N(ATA) = N(A) => xi EN(ATA). So, for a: E $90111, ATAa: = ATA(:E + mi) = ATAa‘: + ATAar:L = Alb. Roy Smith, ECE 210a, 4: 3 Normal equations Solving: Az = b To get a unique solution we need, b e 7?, (A) and N (A) = 0. In this case, 92 = (1434)"1 AT b. Note that ATA 6 RM" and rank(ATA) = rank(A) = n if N (A) = 0. So the inverse (ATA)71 exists. Inconsistent case: b gé R(A) But for the normal equations, ATAm = ATb, ATb e R (AT) = R (ATA) so the normal equations are always consistent. Furthermore, if N (A) = 0, then, a: = (AT/1fl ATb, uniquely solves the normal equations. How does this relate to the original equation: Ax = b? Roy Smith, ECE 210a, 4: 4 Least squares Least squares: The “best” solution to A9: = b. Define 6(.’E) 2: Am — b (residual). If Ax = b is consistent then 23501,. gives 423501“) = 0. But in general we want to minimize some measure of 6. Sum of squares: 2 672(1)) = e(z)Te(w) 2 (AT — b)T(A:z — b) = TTATAT — 2TTATb + bTb 2: f(z) i:l Objective: min f (:11) (note that f (:13) is quadratic) Calculus approach: Differentiate w.r.t. a: and set equal to zero. 6f(w) am) a?“ _ 0 aw am) 01),, Roy Smith, ECE 210a, 4: 5 Least squares Least squares aflx) _ axT T T T 836 317T T 333g — 823A Acc+x A Aami—2axiA b Using the fact that, 393 _ - 8f(x) _ T T T T 6—93 — 6, gives, 8567' — 2e,A Ax 26,,A b. Stacking the 6 f (:17) / 6% into a vector gives, am) = 2ATAx — 2ATb 6m ‘ Now equating this to zero implies that every minimizing solution satisfies, ATAx = ATb, «— Normal equations and the minimum achieved is: min f (x) = f (Tom) 2 min (xTATAx — ZTTATb + bTb) = bTb — xTATb. Roy Smith, ECE 210a, 4: 6 Least squares Converse: All solutions of the normal equations minimize the LS error. Say $50111 is a solution of the normal equations, ATAacmln = ATb. Now consider any other at E ’R” and define: 61., = w — xsoln- Then, ft”) = (61‘ + $90111)TATA(6.L‘ + 1750111) — 2(61: + $50111)TATb + bTb 63.14346,r + 6fATAwmln + xT ATA6m + xT ATAxmln — zafATb — ZacmlnATb + 1% solu soln = f(ms(,1n) + 61TATA6J; (so 9350111 minimizes f ((10)) W Z 0 We can also see that 6fATA6J, = 0 implies that 613 E N (A) = N (ATA) which in this case means that 6:1. and x are also solutions of the the normal equations. Conclusion: z is a LS solution 4:) z solves the normal equations. Caveat Emptor: For theoretical use only! ATAx = ATb is much more likely to have numerical problems than A2: = b. Roy Smith, ECE 210a, 4: 7 Least squares Least squares example: linear (afiine) data fitting b b = + t Data: (t1, ()1) 01 5 (752,52) (tin: bm) t1 t Objective: Find a and fl so that t3 t4 t5 ’56 b = oz + fit approximates the data. 1 t1 171 Linear fitting equations: 1 t 5 = b 771, \ ( 777, W {I} V A b Roy Smith, ECE 210a, 4: 8 Least squares Least squares: Fitting polynomials to data Polynomial form: b = a0 + alt + agt2 + - - - + a,,,1t”’1 The matrix formulation is now: 1 t1 t? tlfil 010 I71 1 t2 t3 télil Oil b2 1 13,, t3, t;;,f1 01",, b,,, —,—1 W V A a; b In this case A is a Vandermonde matrix (see 2.8). If n g m (the number of data points is greater than or equal to the order of the polynomial) and t,- 7é t,- ifi aéj then N(A) = 0. Therefore the LS solution is unique and (theoretically) given by, $0,, = (AT/0’1 ATb. Roy Smith, ECE 210a, 4: 9 Similarity and bases Coordinate matrix representations (see 2.16) Given two sets of basis vectors of the same dimension, BX = {x1,...,:c,,} and By = {y1,...,y,,}, Where X = span(BX) and 3} = span(By). The coordinate matrix representation of an operator: T : X —> 3} is, 15lele = [lT($1)lBy [T(m2)l5y [T($7I)lb’y] In the case of an identity operator, [T(x,-)] By = [(1);]By. We can define the operator, P, P I: By [1131” The matrix, P, effectively changes the coordinates of a vector in the basis BX to its coordinates in the basis By. [vi/5y : Pllea” Roy Smith, ECE 210a, 4: 10 Similarity and bases Change of basis Note that P is non-singular and unique. P’1 maps a vector in the basis By to its coordinates in the basis BX. Matrix operators Say we have an operator, A, defined in terms of the By basis, [UlBy = [AlBy [wlBy- To express this in another basis, apply the change of basis matrix to each of the vectors, 1m = P 1111 = 1A1 P1w1 so, llex = P71 [Alla-P [“1le- and we can identify this as the operator in the basis By. [Ale = P71[A]ByP. Roy Smith, ECE 210a, 4: 11 Similarity and bases Similarity transforms Matrices A and B are similar if there exists an invertible matrix Q such that, A = Q’lBQ. The change of basis operation is a similarity transform on the operator’s coordinate matrix. Invariance under similarity The following properties (amongst others) are preserved by the similarity transform, A = 624362. — rank(A) = rank(B) — Eigenvalues(A) = eigenvalues(B) — trace (A) = trace (B) The trace of a matrix is defined as, trace (A) = Z (1,, (sum of the diagonal elements) {:1 Roy Smith, ECE 210a, 4: 12 Example: Discrete Fourier Transform (DFT) Usual interpretation: sampling a time-domain signal: wk 2 x(t)|t:krr, k = 0, . . . , N — 1, (T is the sample period) The DFT is defined as: N—l 7.1271» X(i) = Zx(k)e , i=0,...,N—1, k:0 and the inverse DFT is given by: Nil 111(k) = %ZX(i)eJ—, k=0,...,N—1. i:[) 72W Simplifying notation: 5 := e V Ni] X(z') = 295005“, i=0,...,N—1, k:0 1N71 w(k) = NZXW’“: k=0,...,N—1. i:(J DFT Express this as a matrix operation: X(0) 1 1 1 1 X0) 1 € £2 6H X<2> = 1 52 54 5H X(N—1) 1 éiV:Hl 8TH? E —,_/ EN The Fourier matrix, FN, has the properties: _ EV E CINXN; — Ev is a Vandermonde matrix (see 2.8); — PEI exists. — others to be discussed later... $(0) 03(1) 03(2) z(N:— 1) Similarity and bases Roy Smith, ECE 210a, 4: 13 Similarity and bases It can be viewed as a change of basis between the time-domain representation, and a discrete-frequency representation. Roy Smith, ECE 210a, 4: 14 ...
View Full Document

{[ snackBarMessage ]}