Classification: Supervised learning, and Model Evaluation
Classifier KNN, DT
Feb 2011
Tommy W. S. Chow
4/14/11
K Nearest Neighbors
lK
l
Nearest Neighbors (KNN)
Advantage
l l l l
Nonparametric architecture Simple Powerful Requires no training time Memory i
Week 2 : Neural Networks
Tommy W S Chow
August 2015
9/8/2015
1
Autonomous Car
from 2007 DARPA Challenge to Google Car
2 million US price money at 2007
went to CMU Boss
https:/www.youtube.com/watch?v=tXuC6gUO4mY
Few years on, Google further improve
the tec
\Ale, ml dame, barkpwnk M
Example 3.1 In a certain region of Russia, the probability that a person lives at least
80 years is 0.75, and the probability that he or she lives at least 90 years is 0.63. What is
the probability that a randomly selected 80
Informatics and Learning Systems
Week 1:
Introduction & Basic Probability
Tommy W S Chow
EE4146
Aug 2015
Text Book: No, Reference book: no
But if you do want to get a copy, try
1. The elements of statistical Learning: data mining, inference, and predictio
X1
X2
1
2.0
3.0
2
4.0
10.0
3
7.0
12.0
4
5.0
8.0
5
1.0
6.0
Find the first and second principle components Y1, Y2.
Solution
Step1: Compute the mean of X1 and X2
mean_X1=(2+4+7+5+1)/5=3.8
mean_x2=(3+10+12+8+6)/5=7.8
Compute the mean data
X1=X1-mean_X1=(2-3.8
Why Eigenvalues
Tommy W S Chow
Dept of EE
CityU
2013
1
Computation analysis of PCA (1) why Eigenvalues?
Given a sample of n observations on a vector of p
variables x1 , x2 , xn p
Define the first principal component of the sample
by the linear transformat
Principal Components
Analysis (PCA)
Tommy W S Chow
Dept of EE
CityU
August 2015
9/16/2015
1
Outline
What is feature reduction and Why?
Feature reduction algorithms
What is PCA?
Basic definitions
Computation analysis of PCA
Examples and applications
Summar
Tutorial on Fishers LDA
Compute the linear discriminant projection for the following twodimensional dataset. List the transformed X1, and X2 numerically in the
transformed axis.
X1=(x1, x2)=cfw_(5,4), (9,9), (1,2)
X2=(x1, x2)=cfw_(10,5), (8,10), (3,9)
Nee
Classification: Supervised learning,
and LDA
Introduction to Discriminant Analysis
(DA), Classification, and Linear DA
Tommy W. S. Chow
January 2015
9/8/2015
1
Recall PCA
In PCA, the main idea to re-express the available dataset to extract the
relevant i
EE4146 Industrial Informatics Weka3.6.1 Users Guide
Weka3.6.1 Users Guide
1. Enter Weka environment from start menu.
Main interface of Weka3.6.1 is shown as the right figure 1
Weka3.6.1 Users Guide
2. Click Explorer
Weka3.6.1 Users Guide
Explorer interfac
Naive Bayes Classifiers
Tommy W. S. Chow Dept. of Electronic Engineering
Classification Methods
Supervised learning of a document-label assignment function Many systems partly rely on machine learning (Google, MSN, Yahoo!, )
Naive Bayes (simple, common
Classification: Supervised learning, and Model Evaluation
Introduction to Discriminant Analysis and Classification
Tommy W. S. Chow
4/14/11
11
What is classification
4/14/11
22
Illustrating Classification Task
i.e., Classify if a man is a business man or
Clustering TWSChow
Jan2010
We must acknowledge Carnegie Mellon University as some of these slides are modified from the CMU AI Clustering ppt
Outline
What is Clustering?
Distance Measure General Applications of Clustering Major Clustering Approaches
2
W
Clustering TWSChow
Jan2010
We must acknowledge Carnegie Mellon University as some of these slides are modified from the CMU AI Clustering ppt
Outline
What is Clustering?
Distance Measure General Applications of Clustering Major Clustering Approaches
2
W
Decision Trees
Find out when people decide to play golf
Outlook sunny sunny overcast rain rain rain overcast overcast sunny sunny Humidity >70 >70 >70 >70 <=70 <=70 <=70 <=70 <=70 <=70 Windy False True False False False True True False False True Play Do
Industrial Informatics EE4146
T W S Chow
Teaching material: Blackboard Text Book: No, Reference book: no But if you do want to get a copy, try
1. AI A modern Approach by Russell & Norvig, Prentice Hall, 2003, 2. Neural Networks and Computing: Learning Alg
Introduction to Graph
Tommy W S Chow
January 2011
Reference book: Introduction to Graph Theory by Gary Chartrand, and Ping Zhang, McGraw Hill
Graph-based representations
l
l
Representing a problem as a graph can provide a different point of view Represent
Graph_2: Minimum Spanning Tree & Minimum path finding
Graph_2 Tommy W S Chow
Jan 2011
1
The Traveling Salesman Problem (TSP) - definition Given : A finite set of points(cities) V and the costs(distances) Cij between each pair of points i,j \in V A tour :
Principal Components Analysis (PCA)
Click to edit Master W S Chow subtitle Tommy Dept of EE style
CityU 2011
Outline
What is feature reduction and Why? n Feature reduction algorithms n What is PCA? n Basic definitions n Computation analysis of PCA n Examp
Principal Components Analysis (PCA)
Click to edit Master W S Chow subtitle Tommy Dept of EE style
CityU 2011
Outline
What is feature reduction and Why? n Feature reduction algorithms n What is PCA? n Basic definitions n Computation analysis of PCA n Examp
Classification: Supervised learning, and Model Evaluation
Introduction to Discriminant Analysis and Classification
Tommy W. S. Chow
What is classification
Illustrating Classification Task
Tid 1 2 3 4 5 6 7 8 9 10
10
Attrib1 Yes No No Yes No No Yes No No N
Assignment 2 10% (can score up to 13%)
Due date: Week 3 class.
You can choose question 1 or question 2
Q1 is a computation NN question which I hope most students will choose to work on. In case students
dislike this type of computational question, you can