III OrthogonalityIII.1 Orthogonality and ProjectionsIII.1.1 Orthogonal vectorsRecall that the dot product, or inner product of two vectorsx=26664x1x2...xn37775y=26664y1y2...yn37775is denoted byx·yorhx,yiand defined byxTy=⇥x1x2· · ·xn⇤26664y1y2...yn37775=nXi=1xiyiSome important properties of the inner product are symmetryx·y=y·xand linearity(c1x1+c2x2)·y=c1x1·y+c2x2·y.The norm, or length, of a vector is given bykxk=px·x=vuutnXi=1x2iAn important property of the norm is thatkxk= 0 implies thatx=0.The geometrical meaning of the inner product is given byx·y=kxkkykcos(✓)where✓is the angle between the vectors. The angle✓can take values from 0 to⇡.The Cauchy–Schwarz inequality states|x·y| kxkkyk.It follows from the previous formula because|cos(✓)|1. The only time that equality occursin the Cauchy–Schwarz inequality, that isx·y=kxkkyk, is when cos(✓) =±1 and✓is either0 or⇡. This means that the vectors are pointed in the same or in the opposite directions.66
III.1 Orthogonality and ProjectionsThe vectorsxandyare orthogonal ifx·y= 0. Geometrically this means either that oneof the vectors is zero or that they are at right angles. This follows from the formula above,since cos(✓) = 0 implies✓=⇡/2.Another way to see thatx·y= 0 means that vectors are orthogonal is from Pythagoras’formula. Ifxandyare at right angles thenkxk2+kyk2=kx+yk2.Butkx+yk2= (x+y)·(x+y) =kxk2+kyk2+ 2x·yso Pythagoras’ formula holdsexactly whenx·y= 0.To compute the inner product of (column) vectorsXandYin MATLAB/Octave we usethe formulax·y=xTy. Thus the inner product can be computed usingX’*Y. (IfXandYare row vectors, the formula isX*Y’.)The norm of a vectorXis computed bynorm(X). In MATLAB/Octave inverse trig functionsare computed withasin(), acos()etc. So the angle between column vectorsXandYcouldbe computed as> acos(X’*Y/(norm(X)*norm(Y)))III.1.2 Orthogonal subspacesTwo subspacesVandWare said to be orthogonal if every vector inVis orthogonal to everyvector inV. In this case we writeV?W.In this figureV?Wand alsoS?T.67
III OrthogonalityA related concept is the orthogonal complement. The orthogonal complement ofV, denotedV?is the subspace containing all vectors orthogonal toV. In the figureW=V?butT6=S?sinceTcontains only some of the vectors orthogonal toS.If we take the orthogonal complement ofV?we get back the original spaceV: This iscertainly plausible from the pictures. It is also obvious thatV✓(V?)?, since any vector inVis perpendicular to vectors inV?