Coursehero >> Pennsylvania >> UPenn >> STAT 553

Course Hero has millions of student submitted documents similar to the one below including study guides, homework solutions, papers, exam answer keys and textbook solutions.

553: Statistics Machine Learning Mikhail Traskin University of Pennsylvania The Wharton School Department of Statistics January 13, 2009 Instructor Instructor: Mikhail Traskin E-mail: mtraskin@wharton O ce: 466 Jon M. Huntsman Hall Recommended Text [HTF] Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, 2001. Other Texts [AB] Martin Anthony and Peter L. Bartlett. Neural network learning: theoretical foundations. Cambridge University Press, 1999. [CBL] Nicol` Cesa-Bianchi and G bor Lugosi. Prediction, Learning, and Games. Cambridge o a University Press, 2006. [DGL] Luc Devroye, L szl Gy r , and G bor Lugosi. A Probabilistic Theory of Pattern a o o a Recognition.Springer-Verlag, 1996. [SS] Bernhard Sch lkopf and Alexander J. Smola. Learning with Kernels. The MIT Press, o 2002. [STC] John Shawe-Taylor and Nello Cristianini. Support Vector Machines and other kernelbased learning methods. Cambridge University Press, 2000. Course Description This course gives a broad overview of the machine learning and statistical pattern recognition. Some topics will be rather glanced over while others will be considered in-depth. Topics include supervised learning (generative/discriminative models, parametric/nonparametric, 1 Spring 2009 Statistics 553 Syllabus 2 neural networks, support vector machines, boosting, bagging, random forests), online learning (prediction with expert advice), learning theory (VC dimension, generalization bounds, bias/variance trade-o ), unsupervised learning (clustering, k-means, PCA, ICA). Most of the course concentrates on the supervised learning and on online learning. Course Requirements There will be regular homework assignments and a nal project. Computing Software Familiarity with Matlab, R, Splus or a related matrix-oriented programming language will be necessary. The open source statistical computing software R is the preferred choice for the course. R can be downloaded from http://www.r-project.org/. Course Prerequisites The prerequisites are the previous course work in linear algebra, multivariate calculus and basic probability and statistics. Familiarity with the optimization theory is helpful but not required. Syllabus Lecture 1 Topics: Supervised, semi supervised and unsupervised learning. Prediction problem. Classi cation vs. regression. Linear regression and classi cation. Logistic regression. Text: HTF, chapters 1 4; DGL. Lecture 2 Topics: Classi cation and the Perceptron Algorithm. Text: HTF, chapter 4. Lecture 3 Topics: Perceptron risk bounds. Text: AB, chapters 4 and 5. Lecture 4 Topics: Kernel classi cation. Kernels and constrained optimization. Text: HTF, chapter 12; chapters STC, 5 and 6. Lecture 5 Topics: Constrained optimization and non-separable SVMs. Text: HTF, chapter 12; STC, chapters 5 and 6. Lecture 6 Topics: Constrained optimization and non-separable SVMs. Text: HTF, chapter 12; STC, chapters 5 and 6. Spring 2009 Statistics 553 Syllabus 3 Lecture 7 Topics: Discrete (0 1) loss and convex loss. SVM loss. Text: Peter L. Bartlett, Michael I. Jordan, and Jon D. McAuli e. Convexity, classi cation, and risk bounds. Journal of the American Statistical Association, 101(473):138 156, 2006. Lecture 8 Topics: Reproducing kernel Hilbert spaces. Text: HTF, chapter 12; STC, chapter 3. Lecture 9 Topics: Representer theorem. Constructing kernels. Text: HTF, chapter 12; STC, chapter 3. Lecture 10 Topics: Kernel methods for regression. Text: HTF, chapter 12; STC, chapter 6. Lecture 11 Topics: Ensemble methods. Text: HTF, chapter 9. Lecture 12 Topics: Ensemble methods and AdaBoost. Text: HTF, chapter 10. Lecture 13 Topics: AdaBoost risk bounds. Text: HTF, chapter 10 Yoav Freund and Robert E. Schapire. Experiments with a new boosting algorithm. In 13th International Conference on Machine Learning, pages 148 156, San Francisco, 1996. Morgan Kaufman. Peter L. Bartlett and Mikhail Traskin. Adaboost is consistent. Journal of Machine Learning Research, 8:2347 2368, 2007. Lecture 14 Topics: Bagging and Random Forests. Text: papers by Leo Breiman. Lecture 15 Topics: Bias, variance and model complexity. Bias-variance decomposition. Text: HTF, chapter 7. Lecture 16 Topics: Model selection: AIC, BIC and MDL. Text: HTF, chapter 7. Lecture 17 Topics: Model selection: Vapnik-Chervonenkis dimension and risk bounds. Text: HTF, chapter 7; AB, chapter 3. Lecture 18 Topics: Model selection: empirical risk minimization and structural risk minimization. Text: HTF, chapter 7; AB, chapter 15. Spring 2009 Statistics 553 Syllabus 4 Lecture 19 Topics: Unsupervised learning. Clustering problems. K-means algorithm and Gaussian mixtures. Text: HTF, chapter 13. Lecture 20 Topics: Gaussian mixtures and EM algorithm. Text: HTF, chapter 8. Lecture 21 Topics: Hidden Markov models. Text: HTF, chapter 8. Lecture 22 Topics: PCA. ICA. Text: HTF, chapter 14. Lecture 23 Topics: Online learning and concept of regret. Prediction with expert advice. Text: CBL, chapters 1 and 2. Lecture 24 Topics: Tight bounds for certain losses. Follow the best expert. Text: CBL, chapter 3. Lecture 25 Topics: Tight bounds for certain losses. Exp-concave loss functions. The greedy forecaster. Text: CBL, chapter 3. Lecture 26 Topics: E cient forecasters for large classes of experts. Text: CBL, chapter 4. Lecture 27 Topics: Sequential investment. Universal portfolios. Text: CBL, chapter 10. Lecture 28 Final project presentation.

Find millions of documents here - Study Guides, Homework Solutions, Papers, Exam Answer Keys and more. Course Hero has millions of course related materials that will enable you to learn better, faster and get an A in all your courses.
Below is a small sample set of documents: