Assignment 2 10% (can score up to 13%)
Due date: Week 3 class.
You can choose question 1 or question 2
Q1 is a computation NN question which I hope most students will choose to work on. In case students
dislike this type of computational question, you can
Classifiers: KNN
and Decision Tree
Sept 2016
12/17/16
1
Different Classifiers
Nearest
Neighbor
Decision
Tree
Linear
Functions
Nonlinear
Functions
g ( x) wT x b
2
This slide is courtesy of www1.cs.columbia.edu/~belhumeur/courses/biometrics/2010/svm.ppt
K N
Naive Bayes Classifiers
Tommy W. S. Chow
Dept. of Electronic Engineering
August 2015
9/8/2015
1
9/8/2015
2
Classification Methods
Supervised learning of a document-label assignment
function
Many systems partly rely on machine learning
(Google, MSN, Yahoo
Tutorial - Nave Bayes - Solutions
Problem: using Naive Bayes Classifier in the application of fault diagnose. The training cases are
as the following table. Diagnose the three cases in test set based on the training set below:
Test set:
case A: Pressure=L
Tutorial: clustering - solutions
Exercise 1:
Use K-means algorithm to cluster each of the following data set into 2 clusters.
A1=[-1.5 1.5], A2=[0 5], A3=[-1 0.5], A4=[0.2 2], A5=[2 5], A6=[-2 3]
Initial value of centroids:
C1=[0 0] C2=[1 0]
Exercise 2:
R
Basic idea of Entropy
We would like to develop a usable measure of the information we get
from observing the occurrence of an event having probability p . Our
first reduction will be to ignore any particular features of the event,
and only observe whether
Tutorial: clustering
Exercise 1:
Use K-means algorithm to cluster each of the following data set into 2
clusters.
A1=[-1.5
1.5], A2=[0
5], A3=[-1
0.5], A4=[0.2
2], A5=[2
5],
A6=[-2 3]
Initial value of centroids:
C1=[0 0]
C2=[1 0]
Exercise 2:
Repeat exerci
Tutorial on Fishers LDA
Compute the linear discriminant projection for the following twodimensional dataset. List the transformed X1, and X2 numerically in the
transformed axis.
X1=(x1, x2)=cfw_(5,4), (9,9), (1,2)
X2=(x1, x2)=cfw_(10,5), (8,10), (3,9)
Nee
Decision Trees
Find out when people decide to play golf
Outlook
Humidity
Windy
Play
sunny
>70
False
Dont play.
sunny
>70
True
Dont play.
overcast
>70
False
Play.
rain
>70
False
Play.
rain
<=70
False
Play.
rain
<=70
True
Dont play.
overcast
<=70
True
Dont
Classification: Supervised learning,
and LDA
Introduction to Discriminant Analysis
(DA), Classification, and Linear DA
Tommy W. S. Chow
January 2015
12/17/16
1
Classification: Definition
Given a collection of records (training set )
- Each record contains
Clustering
Tommy W S Chow
Feb 2015
We must acknowledge Carnegie Mellon
University as some of these slides are modified
from the CMU AI Clustering ppt
Outline
What is Clustering?
Distance Measure
General Applications of Clustering
Major Clustering Appr
Classification: Supervised learning,
and LDA
Introduction to Discriminant Analysis
(DA), Classification, and Linear DA
Tommy W. S. Chow
January 2015
9/8/2015
1
Recall PCA
In PCA, the main idea to re-express the available dataset to extract the
relevant i
Tutorial on Fishers LDA
Compute the linear discriminant projection for the following twodimensional dataset. List the transformed X1, and X2 numerically in the
transformed axis.
X1=(x1, x2)=cfw_(5,4), (9,9), (1,2)
X2=(x1, x2)=cfw_(10,5), (8,10), (3,9)
Nee
Principal Components
Analysis (PCA)
Tommy W S Chow
Dept of EE
CityU
August 2015
9/16/2015
1
Outline
What is feature reduction and Why?
Feature reduction algorithms
What is PCA?
Basic definitions
Computation analysis of PCA
Examples and applications
Summar
Why Eigenvalues
Tommy W S Chow
Dept of EE
CityU
2013
1
Computation analysis of PCA (1) why Eigenvalues?
Given a sample of n observations on a vector of p
variables x1 , x2 , xn p
Define the first principal component of the sample
by the linear transformat
X1
X2
1
2.0
3.0
2
4.0
10.0
3
7.0
12.0
4
5.0
8.0
5
1.0
6.0
Find the first and second principle components Y1, Y2.
Solution
Step1: Compute the mean of X1 and X2
mean_X1=(2+4+7+5+1)/5=3.8
mean_x2=(3+10+12+8+6)/5=7.8
Compute the mean data
X1=X1-mean_X1=(2-3.8
Informatics and Learning Systems
Week 1:
Introduction & Basic Probability
Tommy W S Chow
EE4146
Aug 2015
Text Book: No, Reference book: no
But if you do want to get a copy, try
1. The elements of statistical Learning: data mining, inference, and predictio
\Ale, ml dame, barkpwnk M
Example 3.1 In a certain region of Russia, the probability that a person lives at least
80 years is 0.75, and the probability that he or she lives at least 90 years is 0.63. What is
the probability that a randomly selected 80
Week 2 : Neural Networks
Tommy W S Chow
August 2015
9/8/2015
1
Autonomous Car
from 2007 DARPA Challenge to Google Car
2 million US price money at 2007
went to CMU Boss
https:/www.youtube.com/watch?v=tXuC6gUO4mY
Few years on, Google further improve
the tec
Tutorial on Fishers LDA
Compute the linear discriminant projection for the following twodimensional dataset. List the transformed X1, and X2 numerically in the
transformed axis.
X1=(x1, x2)=cfw_(5,4), (9,9), (1,2)
X2=(x1, x2)=cfw_(10,5), (8,10), (3,9)
Nee
Problem: using Naive Bayes Classifier in the application of fault diagnose. The training cases are
as the following table. Diagnose these three cases:
case A: Pressure=Low, Vibration=Normal, V1=High, V2= Normal, V3=Normal, Percentage under
flow>20, System
Week 2 : Neural Networks
Tommy W S Chow
Sept 2016
9/1/2016
1
Autonomous Car
from 2007 DARPA Challenge to Google Car
2 million US price money at 2007
went to CMU Boss
https:/www.youtube.com/watch?v=tXuC6gUO4mY
Few years on, Google further improve
the techn
Principal Components Analysis (PCA)
Click to edit Master W S Chow subtitle Tommy Dept of EE style
CityU 2011
Outline
What is feature reduction and Why? n Feature reduction algorithms n What is PCA? n Basic definitions n Computation analysis of PCA n Examp
Principal Components Analysis (PCA)
Click to edit Master W S Chow subtitle Tommy Dept of EE style
CityU 2011
Outline
What is feature reduction and Why? n Feature reduction algorithms n What is PCA? n Basic definitions n Computation analysis of PCA n Examp
Graph_2: Minimum Spanning Tree & Minimum path finding
Graph_2 Tommy W S Chow
Jan 2011
1
The Traveling Salesman Problem (TSP) - definition Given : A finite set of points(cities) V and the costs(distances) Cij between each pair of points i,j \in V A tour :
Introduction to Graph
Tommy W S Chow
January 2011
Reference book: Introduction to Graph Theory by Gary Chartrand, and Ping Zhang, McGraw Hill
Graph-based representations
l
l
Representing a problem as a graph can provide a different point of view Represent
Industrial Informatics EE4146
T W S Chow
Teaching material: Blackboard Text Book: No, Reference book: no But if you do want to get a copy, try
1. AI A modern Approach by Russell & Norvig, Prentice Hall, 2003, 2. Neural Networks and Computing: Learning Alg
Decision Trees
Find out when people decide to play golf
Outlook sunny sunny overcast rain rain rain overcast overcast sunny sunny Humidity >70 >70 >70 >70 <=70 <=70 <=70 <=70 <=70 <=70 Windy False True False False False True True False False True Play Do
Clustering TWSChow
Jan2010
We must acknowledge Carnegie Mellon University as some of these slides are modified from the CMU AI Clustering ppt
Outline
What is Clustering?
Distance Measure General Applications of Clustering Major Clustering Approaches
2
W