This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Matrix rank
Matrix products: A E Rm”, B E 72””, AB 6 Rm”
rank(AB) = rank(B) — dim(N (A) n R (B)). rank(B) +ra.nk(A)—n S rank(AB) S min{ra.nk(A),ra.nk(B)} dim=r dim=r dim=s
A
A BR(AT) B B
37am) /— MET)
A . B B N(B)
dim=m—r dim=n—r dim=p—s
Roy Smith, ECE 210a, 4: 1
Matrix rank
M ATA and AAT. Claim: rank(ATA) = rank(A) = rank(AT) Proof:
1: ATA) A) — d N(AT)chA) = A)
%,—/
A
A R(ATA) = R(AT)
\Ay 72(AA1) = ’R,(A)
>< ATA = A
AT A M ) NH 2
m
in
J
H
2
:1;
J Roy Smith, ECE 210a, 4: 2 Normal equations Solving: Ax = b The equation, Ax = b is consistent if there exists at least one solution.
Consistent case Clearly, A3: = b is consistent if and only if b E R (A). Choose a particular solution, Ant 2 b. The general solution is the set, 20,011, = {a%+ati I Aci=b, ati EN(A)}
For a: 6 9165011,, Ax = A(§:+a:t) = A:1‘:+AavL = b. Normal equations: ATAa: = ATb Note that these have the same set of general solutions, because
N(ATA) = N(A) => xi EN(ATA). So, for a: E $90111,
ATAa: = ATA(:E + mi) = ATAa‘: + ATAar:L = Alb. Roy Smith, ECE 210a, 4: 3 Normal equations Solving: Az = b
To get a unique solution we need, b e 7?, (A) and N (A) = 0.
In this case,
92 = (1434)"1 AT b.
Note that ATA 6 RM" and rank(ATA) = rank(A) = n if N (A) = 0.
So the inverse (ATA)71 exists.
Inconsistent case: b gé R(A) But for the normal equations, ATAm = ATb,
ATb e R (AT) = R (ATA)
so the normal equations are always consistent.
Furthermore, if N (A) = 0, then,
a: = (AT/1fl ATb,
uniquely solves the normal equations. How does this relate to the original equation: Ax = b? Roy Smith, ECE 210a, 4: 4 Least squares Least squares: The “best” solution to A9: = b.
Deﬁne 6(.’E) 2: Am — b (residual). If Ax = b is consistent then 23501,. gives 423501“) = 0. But in general we want to minimize some measure of 6. Sum of squares: 2 672(1)) = e(z)Te(w) 2 (AT — b)T(A:z — b) = TTATAT — 2TTATb + bTb 2: f(z) i:l Objective: min f (:11) (note that f (:13) is quadratic) Calculus approach: Differentiate w.r.t. a: and set equal to zero. 6f(w)
am) a?“ _ 0
aw am)
01),,
Roy Smith, ECE 210a, 4: 5
Least squares
Least squares
aﬂx) _ axT T T T 836 317T T
333g — 823A Acc+x A Aami—2axiA b
Using the fact that,
393 _  8f(x) _ T T T T
6—93 — 6, gives, 8567' — 2e,A Ax 26,,A b. Stacking the 6 f (:17) / 6% into a vector gives, am) = 2ATAx — 2ATb
6m ‘ Now equating this to zero implies that every minimizing solution satisﬁes, ATAx = ATb, «— Normal equations
and the minimum achieved is: min f (x) = f (Tom) 2 min (xTATAx — ZTTATb + bTb) = bTb — xTATb. Roy Smith, ECE 210a, 4: 6 Least squares Converse: All solutions of the normal equations minimize the LS error. Say $50111 is a solution of the normal equations, ATAacmln = ATb. Now consider any other at E ’R” and deﬁne: 61., = w — xsoln
Then,
ft”) = (61‘ + $90111)TATA(6.L‘ + 1750111) — 2(61: + $50111)TATb + bTb 63.14346,r + 6fATAwmln + xT ATA6m + xT ATAxmln — zafATb — ZacmlnATb + 1% solu soln = f(ms(,1n) + 61TATA6J; (so 9350111 minimizes f ((10))
W
Z 0 We can also see that 6fATA6J, = 0 implies that 613 E N (A) = N (ATA) which in this case
means that 6:1. and x are also solutions of the the normal equations. Conclusion: z is a LS solution 4:) z solves the normal equations. Caveat Emptor: For theoretical use only! ATAx = ATb is much more likely to have
numerical problems than A2: = b. Roy Smith, ECE 210a, 4: 7 Least squares Least squares example: linear (aﬁine) data ﬁtting b
b = + t
Data: (t1, ()1) 01 5
(752,52)
(tin: bm) t1 t
Objective: Find a and ﬂ so that t3 t4 t5 ’56 b = oz + ﬁt approximates the data. 1 t1 171
Linear ﬁtting equations: 1 t 5 = b
771, \ ( 777,
W {I} V
A b Roy Smith, ECE 210a, 4: 8 Least squares Least squares: Fitting polynomials to data Polynomial form: b = a0 + alt + agt2 +    + a,,,1t”’1 The matrix formulation is now: 1 t1 t? tlfil 010 I71
1 t2 t3 télil Oil b2
1 13,, t3, t;;,f1 01",, b,,, —,—1 W V
A a; b In this case A is a Vandermonde matrix (see 2.8). If n g m (the number of data points is greater than or equal to the order of the polynomial)
and t, 7é t, ifi aéj then N(A) = 0. Therefore the LS solution is unique and (theoretically) given by, $0,, = (AT/0’1 ATb. Roy Smith, ECE 210a, 4: 9 Similarity and bases Coordinate matrix representations (see 2.16) Given two sets of basis vectors of the same dimension,
BX = {x1,...,:c,,} and By = {y1,...,y,,}, Where X = span(BX) and 3} = span(By). The coordinate matrix representation of an operator: T : X —> 3} is, 15lele = [lT($1)lBy [T(m2)l5y [T($7I)lb’y] In the case of an identity operator, [T(x,)] By = [(1);]By.
We can deﬁne the operator, P,
P I: By [1131” The matrix, P, effectively changes the coordinates of a vector in the basis BX to its
coordinates in the basis By. [vi/5y : Pllea” Roy Smith, ECE 210a, 4: 10 Similarity and bases Change of basis
Note that P is nonsingular and unique.
P’1 maps a vector in the basis By to its coordinates in the basis BX.
Matrix operators
Say we have an operator, A, deﬁned in terms of the By basis,
[UlBy = [AlBy [wlBy
To express this in another basis, apply the change of basis matrix to each of the vectors,
1m = P 1111 = 1A1 P1w1
so,
llex = P71 [AllaP [“1le
and we can identify this as the operator in the basis By. [Ale = P71[A]ByP. Roy Smith, ECE 210a, 4: 11 Similarity and bases Similarity transforms Matrices A and B are similar if there exists an invertible matrix Q such that, A = Q’lBQ. The change of basis operation is a similarity transform on the operator’s coordinate matrix.
Invariance under similarity The following properties (amongst others) are preserved by the similarity transform, A = 624362. — rank(A) = rank(B)
— Eigenvalues(A) = eigenvalues(B)
— trace (A) = trace (B) The trace of a matrix is deﬁned as, trace (A) = Z (1,, (sum of the diagonal elements)
{:1 Roy Smith, ECE 210a, 4: 12 Example: Discrete Fourier Transform (DFT) Usual interpretation: sampling a timedomain signal: wk 2 x(t)t:krr, k = 0, . . . , N — 1, (T is the sample period) The DFT is deﬁned as: N—l
7.1271» X(i) = Zx(k)e , i=0,...,N—1, k:0 and the inverse DFT is given by: Nil
111(k) = %ZX(i)eJ—, k=0,...,N—1. i:[) 72W Simplifying notation: 5 := e V Ni] X(z') = 295005“, i=0,...,N—1,
k:0
1N71 w(k) = NZXW’“: k=0,...,N—1. i:(J DFT Express this as a matrix operation: X(0) 1 1 1 1
X0) 1 € £2 6H
X<2> = 1 52 54 5H
X(N—1) 1 éiV:Hl 8TH? E
—,_/
EN The Fourier matrix, FN, has the properties: _ EV E CINXN;
— Ev is a Vandermonde matrix (see 2.8);
— PEI exists. — others to be discussed later... $(0) 03(1)
03(2) z(N:— 1) Similarity and bases Roy Smith, ECE 210a, 4: 13 Similarity and bases It can be viewed as a change of basis between the timedomain representation, and a discretefrequency representation. Roy Smith, ECE 210a, 4: 14 ...
View
Full Document
 Spring '08
 Chandrasekara

Click to edit the document details