This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: K“ x / A
Set 1: Basics I Wu“
1 10
f : 3 y : ,4
“5‘2 2
Basic Operations;
Kyle A. Gallivan 11 '2 3”
Department of Mathematics 3" "L ' Z “l 23" I 6 33/ 2 ‘12
,_ 50 104 6
Florida State University
Linear Combination:
Foundations of Computational Math 1 32
‘all 2010 237 T M 2 wﬁ
\ j \ /
1 4 WEE K ﬂ r “tars“; Vectors and Matrices Scalarx‘ and their operations are assumed to be from 1 0
u the ﬁeld of real numbers (R)
61 = 0 82 = 1
n the ﬁeld of complex numbers (C) 0 0
« complex number: a : l} + 17 where 1' here is used to represent the root
of 71 (occasionally we will use j for this but it will be made clear when 0 1
[MS is done) 83 : 0 e = l
 ti and e, arc the real and imaginary parts of a respectively 1 1 ~ com lcx can ugaic d :13 ~ 1
P J c ’7 — the absolute value ()le denoted la] is «air : l/ﬁ? + 77 / W /_ W
Sealars, Vectors and M trices' Definition 1.1. An m X 1:, matrix of scalars from R or C is a two—dimensionally V‘“ _‘~V. ,,' cc ‘ , ,V
v at 7 a \LLlUI’ Is all one dimensionally ordered list of 71 real scalars ordered arrangcmcnt of mu sealers ~ addition of vectors is componentwisc scalar addition _ l , (1H 012 Ozn
— scalar vcc tor product multiplies each component of the vector With the
scalar Q31 0'22 , , (12,,
_ A r
o C” ,, a vector is an one«dirrtcnsionally ordered list of 7: complex scalars
« addition of vectnrx‘ is componentwisc complex scalar addition “'1“ a“? ’ ' ' “m" A scalar vector )ruduct multi lies each com lex com mm of the vector . ‘ , _
E p p p0 The set of m x n maU'lCCS With scalar elements from R xx denoted Rm”
With the complex scalar The set of m x n, matrices with scalar elements from C is denoted C'“ "" X J K / ’; atrix Operations' Matrix scaling A, B 6 Km“ and "y E R: B ;: 7A :: 47 has elements Li] : Wag Matrix addition :1, B\ C E Rmx": C :2: A ~ B : B ~+r A has elements 71'] : Bi] + 0,] operations Matrix Vector Product ' Deﬁnition 1.2. If X
A:(a1 a2 a")ERmn
and the vector J‘ E R”
{t
62
.l‘ :
{n then
Ar, : aiEi + 11252 + > > , + anEn This is the collection of vectors Rm" and the associated scalar ﬁeld and / [Km—ix Operations I lfA € 52’” “‘2. B 6; R'wxwﬁhcn C E 52""“3 is Scalar deﬁnintion: AB has elements 7;, : 2:1Qik/3k]
Matrixvector deﬁnition: C Ab’ ,_, t; : Ab. 22:! ,,,,, 72.3 where ct : Ce;. 1), 2 Be, Outer product tictinition: ’2
v.21 C : .413 z )T ”,5? where at : Aer, (if _ eTB inner product detinmtion: k C' 2 AB hats elements “1:; :. [13h] where b. r Be.. (2,7 : ETA Matrix Operamgl  the matrix product is not commutative o the matrix product is associative
 the matrix product is distributive, iier, A(B + C) : AB + AC 0 All scalars and vectors can be replaced with submatrices of
appropriate dimension to yield block forms of the matrix product ghj Matrix Operations Deﬁnition 1.3. The transpose of A E Rm”, denoted AT, and the hemiitian
transpose of A E C” x" , denoted AH . are the n X m matrices an 021 amt out Get dmt
. CH2 H22 "' 017112 (312 522 ~' Elm)
7 H
A : V . . A 2
an. (12,, w an”, 6m. 62,. ~ , (“rm K Vector Space I Definition 14. Given scalars .75. a set of vectors V. a vector addition operation
I : y +» z for 1:, y. z E, V. and a scalarvvector product operation y : or for
I. y E V and a 43 IF, we have a vector space ifthe followmg properties hold: I, + y z y 4— 1' (ll
tr+y§+z : r»i‘~(_y+z) (2')
x t O“ r ,r (3) r+ (—1,): : (lv (4)
(at/3)! = MM) (5) (a +2 ,5)! = oz: +51: (6)
()(I + y) : (u + (13/ (7')
1,1 1 r (8} / lar and Vector 0 O,‘ ~ art»(l)a prom
3 M r (71% WW8
: (O+l}o+(‘l)a scular0+1:l
: (0a + la) + («1)a propﬁ
: 0n + (1n + (71)a) prop?
: 011 + (a + (71)a) propS
: ()0, + (0) prop4
: 0a propfi lfl ﬂ
Examples I o P" ~ the set of polynomials of degree less than or equal to n — isomorphic to CHI — elements can be written as a linear combination of 71. + l monomials therefore finite dimensional space u 77x 7 the set of polynomials of any degree — any element can be written as a time sum of monomials  inﬁnite dimensional since it is not the same ﬁnite sum size for all vectors
. maria] : {f ; [0,5] is R, ff ﬂownm < 00} — infinite dimensional ~ need concept of convergence to discuss inﬁnite linear combination that {CPFCSCHCS CﬂCll V‘CCKDI \ J 14 \ o The algebraic structure of a vector spaces considers:
e Suha’paecs
« Linear Transformations
 Bases
 Linear independence
0 The algebraic structure of the vector spaces R" and C" is common to all linite dimensional vector spaces We will use R" in most of our discussions
but the results can be adapted to C“ and all other such vector spaces,  By deﬁnition a vector space V is closed under linear combinations, but an
arbitrary subset of the space is not necessarily closod‘ eg.’ a ﬁnite set or the / set of vectors with nonncgative elements. 15 r \ Deﬁnition 1.5. A subset 5 g R" is a subspace if it is closed under linear
combination, Lenny/1,12. , , Matt. 6 Sthcn for any scalars om :1‘_,,,k airi+a212+~~~+otrt~ ES
and in fact the subspace is itself a vector space (and hence all of our results apply
within 8).
Deﬁnition 1.6. Let S L: R" be a subset (ﬁnite or inﬁnite). The set of all linear combinations of vectors in S is called the span ofS and is a subspace. Example 1.]. R" : spanieheg, _ V V , en) k _/ l6 Matrices and Transformations Deﬁnition 1.7. Given A E C"""",consider b : AI for all 1‘ E C". a The span of the columns of A is a subspace of C'" called the range
MA and is denoted R(A).  Since Am: + ﬂy) : 41,417 + ﬂAy. A defines a linear function
FM) : C” a RM) C; C“ c Any linear function F : C“ —v C" has a unique A deﬁning it. l7 Deﬁnition 1.8. The set of vectors .771. V . . , n are linearly independent if
om +  + mu : 0 M a, : 0 for L r l. , , , Ax if this does not hold then the vectors are linearly dependent. Note that: o A set of vectors being linearly dependent implies one of the vectors can be
written as a linear combination of the others e Any set that contains the 0 vector is linearly dependent, x J [8 l l
f ’ l y z 1
1 0
are linearly independent in El
1 l 3
J' i 1 y : 1 z : 5
1 0 1 are linearly depc ndcnt l9 / ”ﬂ Bases
Deﬁnition 1.9. A set of vectors 11,172, , . V ,n 6 S Q R" is a basis for the
subspace S if
 .it, , 3:1, _ , . in are linearly independent,
 £[Ja71(171,112., Wm.) : .5
Note that: o A subspace has many buses but every basis contains k vectors and the unique
integer k is the dimension of the subspace (k : dzm(5))_ o l: : (ltmiS) is the number of degrees of freedom in 5, ie., 5 is essentially
R“ embedded in R'V . Any collection of vectors in S with k + 1 or more vectors is linearly \ ‘ ____/ dependent l Matrix Implications I o Linear independent columns ofA 6 me" a VI 75 0, Ar, ;4 0
0 Linear dependent columns ofA E Cm“ H 33* ¢ 0 3 AI, : O 0 MIA) 2 {I E C’flxlr 2 0} is a subspace called the null space of AV (Also
called the kernel denoted ker(A),l o # of independent columns immersion of 7104) : column rank of A
o {i of independent rows Tdnnension ofR(A) : row rank of A  lib _ A): F; Rl/l] and mnk(A) : n then the linear function deﬁned by A
is «meteoric and onto RM) and z is unique, \_____~_____/ 21 / \
Analytic Properties I ln addition to the algebraic properties discussed so far we can also define
analytic propenies of veetor spaces and the associated linear transformations,
 size
. distance
o angle
These are analyzed via:
. norms o inner products \ J Size and Distance Deﬁnition 1.10. A vector norm, HIM . is a function C” r, R that satisﬁes 0 HI” 2 O and a: : 0 ~'> “I“ : 0 (deﬁniteness) . limit : lolllrll <homogeneny>
~ llr + in 2 Hill + llyll (mangle inequality) We can also deduce
HI 7 yll 2 WIN , llylll mples Vector Norms I Let I (—1 C" With elements ef’r : {.3 [Norm Eﬁvalence I Theorem 1.1. Let M1) and 1/(r) be vector norms then there exist L’OIZSILIIILY, Le, independent aft, 0 > 0 and T > 0 such that 0/41,“) g 21(2) 3 7311(1) 25 Norm Equivalence I In other words‘ for analytical purposes, all norms are equivalent.
Convergence in one vector norm implies convergence in any other. Note that a and T may be dependent on 71. “I'll: S HIHI S ﬁlll‘lh
[Woo S HIH2 S ﬁllrlloo Hxlloo S MINI E "rllIlloo 26 L Matrix Norms  Deﬁnition 1.1L A matrix norm on me" denoted llAll maps €mx" ,., R and satisﬁes ' EH 0 EMA“ :: ﬁat  5m + Bil w: HBli Examples of matrix norms I Let A 6 CM” with elements LII/rifle, : 041'. Mix 1211a; Zilrianl : £13; HA6; n y _ H
HAllw lg?" Eprlaul e 12123,, He. HAM: ' “gag! MAIN: ”Ally ' “gala:1 “Allin ll!
Alil HAHF STABLHUUV : \/ ELM/18:“? Examples of matrix norms o The Frobenius norm HAN]: is essentially the vector the matrix as if it was :1 element of CW”, 2 norm applied to 0 IA 2 : trace AHA; where the trace is the sum of the dia onal
l g elements. a While all matrix norms are equivalent for analytical purposes‘ they differ considerably in their ease of computation, Matrix 2 Norm I 0 Deﬁnition given requires optimization “All; r: max “Arm
llxllzzl . HAM can be related to eigenvalues and singular val also “inﬁnite" computations can be used for approximation [MR2 S x/HAHtllAlioc k ties but these are I Bounds can be derived in terms of “Alix and EMHN. ire“ equivalence ___./ ent Matrix Norms Definition 1.12. The matrix norms H , Hm“ v “[1,” , EL, are consistent if
llABllu S llAllullBlh whenever the product exists:
Lemma 1.1. The matrix 11horm deﬁnes afamt'ly ofconsistent matrix norms.
Specificallij xi E C'"X",B 6 CM” and: E C“ llABllp S llAllpllBlln
llA'Illp S llAllpllIllp 31 Induced Matrix Norms Deﬁnition 1.13. The matrix norm H , H is subordinate to vector norms H ‘ Ho, and
il > lira if llrh'lla S llAllllIllts
and the matrix norm therefore bounds the expansion/contraction of the linear
transtormution deﬁned by A,
Deﬁnition 1.14. Given vector norms H » H0 and H A ”3 me induced matrix norm
ll ' ”an ‘5 HAHN : Ilirliigl llAIllB \_________________/ 32 /—__~—“—\ Induced Matrix Norms ' Theorem 13. Given a Vector norm ﬂ ~ llo on C" or R" the induced matrix norm for an n, x n, mam} Elm (subardinalc) tlollB‘iln (.mbnwllipiicazive) x] / Convergence I Both vector sequences and matrix sequences can therefore be said to
converge to limit vectors and limit matrices by considering convergence
in IR, Deﬁnition 1.15 For the vector sequence {It} and the matrix sequence {At} 11m xk : ;r H lim flrk 11E! : O
k‘wxy kam Componentwise convergence for both follows. Angles in ndimensional Spaces I Deﬁnition 1.16. An inner product (or scalar product) on a vector space V is a
map < 1,: >2 V x V ., F where the ﬁeld F is either R orC mat satisﬁes lv <acc+i3ziy >: a <I,y > +I3< z.y>,WiﬁII,y‘Z E Vand
at} E F.(lineu1ity) 2. < guy >: < 11,1 > (hermitian) 3‘ < 1.1 >2 0 and < 1,: >: 0 H x : (J (deﬁniteness) 35 FInEéFWEEY  < 11y >2: IHy is an inner product for C”
u < .13. y >2 rTy is an inner product for R"
0 There are other inner products for C” and R". . “I“ ;: v’< 1.1 > isanorm, Lemma 1.4. For I. y E C" o iJiﬂyé i glrllyljyéjq wnh l; + i : 1 (Hoelderinequality)
c gxﬂyl f jrligllyﬁz (Cauchy~Sc/zwarz inequality}
 My; < ‘11, ally/Hm Angles can be deﬁned by making the CauchySchwinn inequality an equality,
Definition 1.17. Let a: and y be two nonzero vectoxs in C" then the cosine ofthe
angle between the one~dimensional Spaces deﬁned by the vectors, 0 g (9 g 7r/2,
is deﬁned Wit/l = 6059HIHnHz/H2 \ / 37 Generalization from R2 f—ﬁ Consider 3:. y E R2 positive quadrant. ﬂy : cosélllwllllyll iT'g : C059
.i; z((;osl)1xsin61)andﬂi[l:l
1] : (cos 62,si1102) and Hg)“ :1
where 61 and 62 are angles from (1, 0) ng :: (:05 ()1 (10562 + sin 61 sin 02 : cos(0] ~ (92) : 0050 p_____________J 3B *_\ Orthogonality Deﬁnition 1.18. The vectors .r and y am said to be orthogonal if their inner
product is 0, it: , <: r. y >: rHy .: 0.
This generalizes the Pythagorean Theorem to multidimensional and complex
vectors: 1, , I 2 g , I! , , :zr + W: ,, (I + y) (I + til :2: J‘HI + yHy + 2Relzuy) : VEHI+yuy A Iii”: + lit/Hi Theorem 15. Let V be a vector space over IR (similar statmenlx can be
made for C) with and inner product < 1:, y >. Ifthe norm is deﬁned
byllril : lr< LBthen we have ' lll‘ + UH2 :1 ”I“? + [lyllz + 2 < Ivy >
° 111 , HIV : 111112 + Hyll2 , 2 < Ly > (L‘osinelavv)  Hr + M? + HI 7 W2 : MIN? + llyllzl (parallelogram law} 0 i1]; + yHQ 711$ v sz : 4 < Ly > (polarization identity) \___________/ [Polarization and Parallelograms' Amazingly, the reverse is also true, Theorem 1.6. Let V be a normed vector space over [R (similar xtutments
can be made for (C). If the norm satixﬁes the parallelogram law then the
polarization identity deﬁnes an inner productfor V, That is ill + 9H? + [11‘ , yll) : 9(ll3‘ll2 +Hyl12) ll < m >: inn + y“: 7 HI  W}
U [136+ W = ll'IH2 + “all2 + 2 < Ivy > and HI , y“? : NIH2 + Hyll2 — 2 < ray > 41 ...
View
Full Document
 Spring '11
 Gallivan

Click to edit the document details