Chapter 3.pdf - Chapter 3 MACHINE LEARNING CLASSIFIERS...

Chapter 3
MACHINE LEARNING CLASSIFIERS USING
SCIKIT-LEARN
Sanjay Kumar C K
SANJAY KUMAR C K

Objectives
 An introduction to robust and popular algorithms for classification, such
as logistic regression, support vector machines, and decision trees
 Examples and explanations using the scikit-learn machine learning library,
which provides a wide variety of machine learning algorithms via a user-
friendly Python API
 Discussions about the strengths and weaknesses of classifiers with linear
and nonlinear decision boundaries
SANJAY KUMAR C K

Choosing a classification
algorithm
It is recommended that we compare the performance of at least a handful of different learning algorithms
to select the best model for the particular problem; these may differ in the number of features or
examples, the amount of noise in a dataset, and whether the classes are linearly separable or not .
The performance of a classifier (computational performance as well as predictive power) depends on the
data available for learning.
The five main steps in training a supervised machine learning algorithm are:
1. Selecting features and collecting labeled training examples.
2. Choosing a performance metric.
3. Choosing a classifier and optimization algorithm.
4. Evaluating the performance of the model.
5. Tuning the algorithm.
SANJAY KUMAR C K

Scikit-learn – training a
perceptron
from sklearn import datasets
import numpy as np
iris=datasets.load_iris()
print (iris)
X = iris.data[:, [2, 3]]
y = iris.target
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.3, random_state=1, stratify=y)
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
sc.fit(X_train)
X_train_std = sc.transform(X_train)
X_test_std = sc.transform(X_test)
from sklearn.linear_model import Perceptron
ppn = Perceptron(eta0=0.1, random_state=1)
ppn.fit(X_train_std, y_train)
y_pred = ppn.predict(X_test_std)
print('\nMisclassified examples: %d' % (y_test !=
y_pred).sum())
from sklearn.metrics import accuracy_score
print('Accuracy: %.3f' % accuracy_score(y_test, y_pred))
SANJAY KUMAR C K

SANJAY KUMAR C K

Modeling Class probabilities using
Logistic Regression
logistic regression is a probabilistic model for binary
classification
when p stands for the probability of the positive event,
we define odds as:
We can then further define the logit function, which is
simply the logarithm of the odds (log-odds):
We define the conditional probability, p(y=1|x) that a particular example belongs to class 1 given its features, x .
we are interested in predicting the probability that a certain example belongs to a particular class, which is the inverse
form of the logit function, called the logistic sigmoid function .
SANJAY KUMAR C K

Logistic Regression
sigmoid function:
Here, z is the net input, the linear combination of weights, and
the inputs
SANJAY KUMAR C K