EECS 598-005: Theoretical Foundations of Machine Learning
Fall 2015
Lecture 2: Convex Analysis
Lecturer: Jacob Abernethy
Scribes: Ning Jiang
Disclaimer: These notes have not been subjected to the usual scrutiny reserved for formal publications.
2.1
A few

EECS 598-005: Theoretical Foundations of Machine Learning
Fall 2015
Lecture 8: PAC Guarantee for Infinite Sets and Growth Function
Lecturer: Jacob Abernethy
8.1
Scribes: Yike Liu
Editors: Luke Brandl, Nghia Nguyen, and Shuang Qiu
Review: Coin Toss
Recall

EECS 598-005: Theoretical Foundations of Machine Learning
Fall 2015
Lecture 19: Fast Rates 1
Lecturer: Matus Telgarsky, Scribes: Cupjin Huang, Editors: Biaoshuai Tai, Wei Lee, Shang-En Huang
Notations Notations in this lecture are pretty convoluted. In ge

EECS 598-005: Theoretical Foundations of Machine Learning
Fall 2015
Lecture 9: Relations between Lectures and Sauers Lemma
Lecturer: Jacob Abernethy
9.1
Scribes: Hsu Kao, Editor: Jinqi Shen
Relations between Lectures
Here is the Big Question: When does le

EECS 598-005: Theoretical Foundations of Machine Learning
Fall 2015
Lecture 11: Margin Theory
Lecturer: Jacob Abernethy
11.1
Scribe: Daniel LeJeune, Editors: Pengyu Xiao
Review: Lower Bounds
Theorem 11.1. Given any class C with VC-dimension d, there exist

EECS 598-005: Theoretical Foundations of Machine Learning
Fall 2015
Lecture 13: Rademacher Complexity and Massarts Lemma
Lecturer: Jacob Abernethy
13.1
Scribes: Yi-Jun Chang, Editors: Yitong Sun and David Hong
Rademacher Complexity
Given a function class

EECS 598-005: Theoretical Foundations of Machine Learning
Fall 2015
Lecture 16: Perceptron and Exponential Weights Algorithm
Lecturer: Jacob Abernethy
Scribes: Yue Wang, Editors: Weiqing Yu and Andrew Melfi
16.1
Review: the Halving Algorithm
16.1.1
Proble

EECS 598-005: Theoretical Foundations of Machine Learning
Fall 2015
Lecture 4: Hoeffdings Inequality and Martingales
Lecturer: Jacob Abernethy
Scribes: Ruihao Zhu, Editors: Yuan Zhuang
Disclaimer: These notes have not been subjected to the usual scrutiny

Theoretical Foundations of Machine Learning - Solutions #4
Written by Chansoo Lee
Due:
1) LFP.
Consider Perceptron with weight w on the repetition of the sequence (a1 , 1), . . . , (am , 1). Then, w correctly
classifies everything to be positive if and on

EECS 598-005: Theoretical Foundations of Machine Learning
Fall 2015
Lecture 12: Noisy Setting and Rademacher Complexity
Lecturer: Jacob Abernethy
12.1
Scribes: Aniket Deshmukh
Noisy Setting
In the most general setting, the hypothesis class H may contain n

EECS 598-005: Theoretical Foundations of Machine Learning
Fall 2015
Lecture 5: Typical Machine Learning Problem and PAC Learning
Lecturer: Jacob Abernethy
5.1
Scribes: Frank Cheng
Editors: Luke Brandl, Nghia Nguyen, and Shuang Qiu
Review: Hoeffdings inequ

EECS 598-005: Theoretical Foundations of Machine Learning
Fall 2015
Lecture 22: Adversarial Multi-Armed Bandits
Lecturer: Jacob Abernethy
22.1
Scribes: Arvind Prasadan
The EXP3 Algorithm1
EXP3 was invented in 2001 by Auer, Cesa-Bianchi, Freund, and Schapi

EECS 598-005: Theoretical Foundations of Machine Learning
Fall 2015
Lecture 18: Nashs Theorem and Von Neumanns Minimax Theorem
Lecturer: Jacob Abernethy
Scribes: Lili Su, Editors: Weiqing Yu and Andrew Melfi
18.1
Review: On-line Learning with Experts (Act

EECS 598-005: Theoretical Foundations of Machine Learning
Fall 2015
Lecture 14: Growth Function Generalization Bound and Online Learning
Lecturer: Jacob Abernethy
14.1
Scribes: Shun Zhang
Growth Function Generalization Bound
Recall Massarts Lemma from the

EECS 598-005: Theoretical Foundations of Machine Learning
Fall 2015
Lecture 20: Fast Rates, Part II
Lecturer: Matus Telgarsky, Scribes: Mark Heimann, Editors: Biaoshuai Tai, Wei Lee, Shang-En Huang
20.1
Setup
Assume we are given the following:
Parameter

EECS 598-005: Theoretical Foundations of Machine Learning
Fall 2015
Lecture 7: General PAC Guarantee
Lecturer: Jacob Abernethy
7.1
Scribes: Xi Liu
Editors: Luke Brandl, Nghia Nguyen, and Shuang Qiu
Review: Learning Axis-Aligned Rectangles
X = R2 , Y = cfw

EECS 598-005: Theoretical Foundations of Machine Learning
Fall 2015
Lecture 3: Concentration Inqualities
Lecturer: Jacob Abernethy
Scribes: Yuchen Jiang, Editors: Yuan Zhuang
Disclaimer: These notes have not been subjected to the usual scrutiny reserved f

EECS 598-005: Theoretical Foundations of Machine Learning
Fall 2015
Lecture 6: Risk and PAC-Learnable Class
Lecturer: Jacob Abernethy
6.1
Scribes: Sentao Miao
Editors: Luke Brandl, Nghia Nguyen, and Shuang Qiu
Error Bound and Sample Size Requirement
Examp

Theoretical Foundations of Machine Learning - Homework #1
Your Name (uniqname), Collaborators: Person 1, Person 2, First Name Suffices
Due: 9/28/2015
Homework Policy: Working in groups is fine. Please write the members of your group on your solutions.
The

Theoretical Foundations of Machine Learning - Solutions #3
Written by David Hong, Chansoo Lee, and Shuang Qiu
Due: 10/30/2015
1) Rademacher Complexity.
(a) Note that Rademacher distribution is symmetric and hence we can flip the sign of inside expectation

Theoretical Foundations of Machine Learning - Homework #2
Jacob Abernethy and Chansoo Lee
Due: 10/16/2015
Homework Policy: Working in groups is fine, but every student must submit their own writeup. Please
write the members of your group on your solutions

EECS 598-002
Hybrid Systems: Specification, Verification and Control
Fall 2015
Lecture Notes 2
Necmiye Ozay, 9/16/2015
These notes are mostly based on chapters 2 and 3 of [1].
1
Transition systems
A proposition is a statement that can either be true or fa

EECS 598-002
Hybrid Systems: Specification, Verification and Control
Fall 2015
Lecture Notes 8
Necmiye Ozay, 10/5/2015
These notes are mostly based on chapter 5.1 of [1].
1
Linear temporal logic
A linear-time property P over a set AP of atomic proposition

EECS 598-002
Hybrid Systems: Specification, Verification and Control
Fall 2015
Lecture Notes 1
Necmiye Ozay, 9/14/2015
1
Propositional logic: a quick review
To be written later. See appendix A.3 of Baier and Katoen.
2
Verification and control synthesis
In

EECS 598-002
Hybrid Systems: Specification, Verification and Control
Fall 2015
Lecture Notes 5
Necmiye Ozay, 9/28/2013
These notes are mostly based on chapter 4 of [1]. A good reference for finite automata
and regular languages is [2].
1
Linear-time prope

EECS 598-002
Hybrid Systems: Specification, Verification and Control
Fall 2015
Lecture Notes 9
Necmiye Ozay, 10/7/2015
These notes are mostly based on sections 2.1-2.2-2.3 of [1].
1
Introductory concepts in convex geometry
In reachability analysis, verifi

EECS 598-002 Fall 2016
Hybrid Systems: Specica>on,
Verica>on and Control
Lecture 3
Necmiye Ozay
Outline:
Course administra6on
More on hybrid system modeling paradigms
Proposi6onal logic review
Transi6on systems (Chapters 2.1-2.2 from Baier an

EECS 598-002
Hybrid Systems: Specification, Verification and Control
Fall 2016
Lecture Notes 1
Necmiye Ozay, 9/14/2016
1
Propositional logic: a quick review
To be written later. See appendix A.3 of Baier and Katoen.
2
Verification and control synthesis
In