Confusion matrix
Predicted Class Label
Actual negative
1
Actual Class Label
Actual positive
0
Predicted positive
1
True positive
False positive
Predicted negative
0
False negative
True negative
There are four possible cases:
For an actually positive sampl

Why not parametric methods?
Parametric approaches require knowing the form of the
density.
E.g. With ML estimation in Lecture 2 we assumed that
the underlying function of our data is a Gaussian.
However, in many cases,
- The form is not known
- The form d

Generative vs. Discriminative
There are two main methods for learning algorithms:
generative and discriminative.
Logistic regression is an example to a learning
algorithm that models p(y|x), i.e. the conditional
(posterior) probability of y given x.
It is

Classification
Logistic Regression is a classification method!
Examples:
Email: Spam / Not Spam?
Brain tumor: Malignant / Benign ?
y cfw_0,1
0: Negative Class (e.g., benign tumor)
1: Positive Class (e.g., malignant tumor)
y cfw_0,1, 2, .
if there are mo

Gaussian (Normal) Distribution
p(x) = N ( , 2)
1
p( x)
e
2
x 2
2 2
: Mean
: Standard deviation: average absolute difference from the mean.
2: Variance: average squared difference from the mean.
Figure from Introduction to Machine Learning 2ed., E Alpay

Diagnosis for Learning Algorithms
Suppose you have implemented linear regression
to predict housing prices.
However, when you test your hypothesis on a new
set of houses, you find that it makes very large
errors in its predictions. What should you try nex

Multiple Variables
By multivariate, we mean there are multiple
variables/features:
Size (feet2) Number of Number of Age of home
bedrooms
floors
(years)
2104
1416
1534
852
5
3
3
2
1
2
2
1
Price ($1000)
46
40
30
36
460
232
315
178
Notation:
= number of feat

Bayes Decision Theory
It is a statistical approach to machine learning.
Assumptions:
- Decision problem is probabilistic
- All relevant probability values are known
We derive decision rules that are optimal in the
sense that they either minimize average p

About the Course
Lecturer: Nesli Erdomu
E-mail: [email protected]
Exams: One midterm (~30%) and one final (~40%)
Assignments/projects: There will be a few programming
assignments (to be completed in Python).
We will have a Python introduction sess