# Final rule basically says that the smallest norm is

• 505

This preview shows page 63 - 65 out of 505 pages.

final rule basically says that the smallest norm is achieved by a matrix or vector consisting ofall zeros. Its possible to define a norm that gives zero norm to nonzero matrices, but you cant give nonzero norm to zero matrices. Thats a mouthful, but if you digest it then you probablyhave grepped the important concepts here.If you remember Euclidean distances (think Pythagorastheorem) from grade school, thennon-negativity and the triangle inequality might ring a bell. You might notice that norms sounda lot like measures of distance.In fact, the Euclidean distancex21+· · ·+x2nis a norm. Specifically its the2-norm. Ananalogous computation, performed over the entries of a matrix, e.g.i,ja2ij, is called theFrobenius norm. More often, in machine learning we work with the squared2norm (notated22). We also commonly work with the1norm. The1norm is simply the sum of the absolutevalues. It has the convenient property of placing less emphasis on outliers.To calculate the2norm, we can just callnd.norm().In [19]:nd.norm(x)Out[19]:[3.7416575]<NDArray 1 @cpu(0)>To calculate the L1-norm we can simply perform the absolute value and then sum over theelements.In [20]:nd.sum(nd.abs(x))Out[20]:[6.]<NDArray 1 @cpu(0)>2.4. Linear algebra57
2.4.12 Norms and objectivesWhile we dont want to get too far ahead of ourselves, we do want you to anticipate why theseconcepts are useful. In machine learning were often trying to solve optimization problems:Maximizethe probability assigned to observed data.Minimizethe distance between predictionsand the ground-truth observations. Assign vector representations to items (like words, prod-ucts, or news articles) such that the distance between similar items is minimized, and the dis-tance between dissimilar items is maximized. Oftentimes, these objectives, perhaps the mostimportant component of a machine learning algorithm (besides the data itself), are expressedas norms.2.4.13 Intermediate linear algebraIf youve made it this far, and understand everything that weve covered, then honestly, youareready to begin modeling. If youre feeling antsy, this is a perfectly reasonable place to moveon. You already know nearly all of the linear algebra required to implement a number of manypractically useful models and you can always circle back when you want to learn more.But theres a lot more to linear algebra, even as concerns machine learning. At some point, ifyou plan to make a career of machine learning, youll need to know more than weve coveredso far. In the rest of this chapter, we introduce some useful, more advanced concepts.Basic vector propertiesVectors are useful beyond being data structures to carry numbers. In addition to reading andwriting values to the components of a vector, and performing some useful mathematical oper-ations, we can analyze vectors in some interesting ways.

Course Hero member to access this document

Course Hero member to access this document

End of preview. Want to read all 505 pages?