Logistic Regression
P R O F. N AV N E E T G O YA L
D E PA R T M E N T O F C O M P U T E R S C I E N C E
BITS-PILANI, PILANI CAMPUS
Logistic Regression
Source of figure unknown
Logistic Regression
Linear Regression relationship between a continuous
respons

HIDDEN MARKOV
MODELS
Prof. Navneet Goyal
Department of Computer Science
BITS, Pilani
Topics
Markov Models
Hidden Markov Models
HMM Problems
Markov Analysis
A technique that deals with the
probabilities of future occurrences by
analyzing presently know

Polynomial Curve Fitting
BITS C464/BITS F464
Navneet Goyal
Department of Computer Science, BITS-Pilani, Pilani Campus, India
Polynomial Curve Fitting
Seems a very trivial concept!
Why are we discussing it in Machine Learning
course?
A simple regression pr

Support Vector Machines
Text Book Slides
Support Vector Machines
Find a linear hyperplane (decision boundary) that will separate the data
Support Vector Machines
One Possible Solution
Support Vector Machines
Another possible solution
Support Vector Mac

Curse of Dimensionality
Prof. Navneet Goyal
Dept. Of Computer Science & Information Systems
BITS - Pilani
Curse of Dimensionality!
Poses serious challenges !
Important factor influencing the design on pattern recognition techniques
Mixture of oil, wate

Machine Learning
Navneet Goyal
Vimal S P
Dept. of CSIS
BITS, Pilani
Introduction
Learning
To
gain knowledge
To gain understanding
To develop skills
By study or instructions or experience
Can be defined as
Modification of a behavioral tendency by
expe

Classification:
Linear Models
Prof. Navneet Goyal
CS & IS
BITS, Pilani
Classification
By now, you are well aware of the classification problem
Assign an input vector x to one of the K discrete disjoint
classes, Ck
Overlapping classes: Multi-label classifi

Bayesian Classification
Dr. Navneet Goyal
BITS, Pilani
Bayesian Classification
What are Bayesian Classifiers?
Statistical Classifiers
Predict class membership
probabilities
Based on Bayes Theorem
Nave Bayesian Classifier
Computationally Simple
Comparable

Reinforcement Learning
Vimal
The learner here is a decision making
agent
Keeps making decisions to take actions in
the environment and receive rewards(or
penalty)
A set of trial-and-error runs, the agent is
expected to learn the best policy that
maximize

Classification with MATLAB
(with PRTools 4.1)
BITS Pilani
Pilani Campus
BITS C464 - Machine Learning
Department of Computer Science and Information Systems
To Discuss
Importing datasets / creating synthetic datasets,
Creating Mapping & Visualizing Results

Linear Regression &
Classification
Prof. Navneet Goyal
CS & IS
BITS, Pilani
Fundamentals of Modeling
Abstract representation of a real-world process
Y=3X+2 is a very simple model of how variable Y might
relate to variable X
Instance of a more general mode

HIDDEN MARKOV
MODELS
Prof. Navneet Goyal
Department of Computer Science
BITS, Pilani
Topics
Markov Models
Hidden Markov Models
HMM Problems
Markov Analysis
A technique that deals with the
probabilities of future occurrences by
analyzing presently know

Classification with MATLAB
(with PRTools 4.1)
BITS Pilani
Pilani Campus
BITS C464 - Machine Learning
Department of Computer Science and Information Systems
To Discuss
Importing datasets / creating synthetic datasets,
Creating Mapping & Visualizing Results

Logistic Regression
P R O F. N AV N E E T G O YA L
D E PA R T M E N T O F C O M P U T E R S C I E N C E
BITS-PILANI, PILANI CAMPUS
Logistic Regression
Source of figure unknown
Logistic Regression
Linear Regression relationship between a continuous
respons

Linear Regression &
Classification
Prof. Navneet Goyal
CS & IS
BITS, Pilani
Fundamentals of Modeling
Abstract representation of a real-world process
Y=3X+2 is a very simple model of how variable Y might
relate to variable X
Instance of a more general mode

Linear Regression &
Classification
Prof. Navneet Goyal
CS & IS
BITS, Pilani
Fundamentals of Modeling
Abstract representation of a real-world process
Y=3X+2 is a very simple model of how variable Y might
relate to variable X
Instance of a more general mode

Machine Learning
Navneet Goyal
Vimal S P
Dept. of CSIS
BITS, Pilani
Introduction
Learning
To
gain knowledge
To gain understanding
To develop skills
By study or instructions or experience
Can be defined as
Modification of a behavioral tendency by
expe

Ensemble Classifiers
Prof. Navneet Goyal
Ensemble Classifiers
Introduction & Motivation
Construction of Ensemble Classifiers
Boosting (Ada Boost)
Bagging
Random Forests
Empirical Comparison
Introduction & Motivation
Suppose that you are a patient w

Linear Classification Models:
Generative
Prof. Navneet Goyal
CS & IS
BITS, Pilani
Approaches to Classification
Probabilistic
Discriminant fn.
Inference stage use training data to learn a model
for p(Ck|x)
Decision stage use posterior probabilities p(C k|x

Polynomial Curve Fitting
BITS C464/BITS F464
Navneet Goyal
Department of Computer Science, BITS-Pilani, Pilani Campus, India
Polynomial Curve Fitting
Seems a very trivial concept!
Why are we discussing it in Machine Learning
course?
A simple regression pr

Reinforcement Learning
Vimal
The learner here is a decision making
agent
Keeps making decisions to take actions in
the environment and receive rewards(or
penalty)
A set of trial-and-error runs, the agent is
expected to learn the best policy that
maximize

Bayesian Classification
Dr. Navneet Goyal
BITS, Pilani
Bayesian Classification
What are Bayesian Classifiers?
Statistical Classifiers
Predict class membership
probabilities
Based on Bayes Theorem
Nave Bayesian Classifier
Computationally Simple
Comparable

Ensemble Classifiers
Prof. Navneet Goyal
Ensemble Classifiers
Introduction & Motivation
Construction of Ensemble Classifiers
Boosting (Ada Boost)
Bagging
Random Forests
Empirical Comparison
Introduction & Motivation
Suppose that you are a patient w

Curse of Dimensionality
Prof. Navneet Goyal
Dept. Of Computer Science & Information Systems
BITS - Pilani
Curse of Dimensionality!
Poses serious challenges !
Important factor influencing the design on pattern recognition techniques
Mixture of oil, wate

Classification:
Linear Models
Prof. Navneet Goyal
CS & IS
BITS, Pilani
Classification
By now, you are well aware of the classification problem
Assign an input vector x to one of the K discrete disjoint
classes, Ck
Overlapping classes: Multi-label classifi

Linear Classification Models:
Generative
Prof. Navneet Goyal
CS & IS
BITS, Pilani
Approaches to Classification
Probabilistic
Discriminant fn.
Inference stage use training data to learn a model
for p(Ck|x)
Decision stage use posterior probabilities p(C k|x

Linear Regression &
Classification
Prof. Navneet Goyal
CS & IS
BITS, Pilani
Fundamentals of Modeling
Abstract representation of a real-world process
Y=3X+2 is a very simple model of how variable Y might
relate to variable X
Instance of a more general mode

A C TI E LEA R N I G
V
N
Navneet Goyal
Slides developed using material from:
1. Simon Tong, ACTIVE LEARNING: THEORY AND APPLICATIONS.
Ph.D. dissertation, Stanford University, August, 2001.
2. Burr Settles. ACTIVE LEARNING LITERATURE SURVEY
.
Computer Scie

Classification:
Linear Models
Prof. Navneet Goyal
CS & IS
BITS, Pilani
Classification
By now, you are well aware of the classification problem
Assign an input vector x to one of the K discrete disjoint
classes, Ck
Overlapping classes: Multi-label classifi

A C TI E LEA R N I G
V
N
Navneet Goyal
I ance Based Lear ng
nst
ni
Rote Classifier
K- nearest neighbors (K-NN)
Case Based Resoning (CBR)
Cl
assi cati
f i on: Eager & Lazy
Learners
Decision Tree classifier is an example of an
eager learner
Because the