Predicted Class Label
Actual Class Label
There are four possible cases:
For an actually positive sampl
Why not parametric methods?
Parametric approaches require knowing the form of the
E.g. With ML estimation in Lecture 2 we assumed that
the underlying function of our data is a Gaussian.
However, in many cases,
- The form is not known
- The form d
Generative vs. Discriminative
There are two main methods for learning algorithms:
generative and discriminative.
Logistic regression is an example to a learning
algorithm that models p(y|x), i.e. the conditional
(posterior) probability of y given x.
Logistic Regression is a classification method!
Email: Spam / Not Spam?
Brain tumor: Malignant / Benign ?
0: Negative Class (e.g., benign tumor)
1: Positive Class (e.g., malignant tumor)
y cfw_0,1, 2, .
if there are mo
Gaussian (Normal) Distribution
p(x) = N ( , 2)
: Standard deviation: average absolute difference from the mean.
2: Variance: average squared difference from the mean.
Figure from Introduction to Machine Learning 2ed., E Alpay
Diagnosis for Learning Algorithms
Suppose you have implemented linear regression
to predict housing prices.
However, when you test your hypothesis on a new
set of houses, you find that it makes very large
errors in its predictions. What should you try nex
By multivariate, we mean there are multiple
Size (feet2) Number of Number of Age of home
= number of feat
Bayes Decision Theory
It is a statistical approach to machine learning.
- Decision problem is probabilistic
- All relevant probability values are known
We derive decision rules that are optimal in the
sense that they either minimize average p
About the Course
Lecturer: Nesli Erdomu
E-mail: [email protected]
Exams: One midterm (~30%) and one final (~40%)
Assignments/projects: There will be a few programming
assignments (to be completed in Python).
We will have a Python introduction sess