RANDOM
FORESTS
S UPPLEMENTAL MATERIAL TO RANDOM FORESTS LECTURE
C SE 455/555
P ROF. J CORSO
MATERIALS ARE TAKEN FROM AMIT AND GEMAN, SHAPE QUANTIZATION AND RECOGNITION
WITH RANDOMIZED TREES, NEURAL COMPUTATION, 9(7):15451588, 1997.
THERE ARE A NUMBER OF P
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 19, NO. 7, JULY 1997
711
Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection
~ Peter N. Belhumeur, Joao P. Hespanha, and David J. Kriegman
AbstractWe develop a
Problem 3: Parametric and Non-Parametric Methods
Figure 1: Histogram
1. The histogram is shown in Figure 1.
2. The kernel density function is dened as
pn (x) =
1
n
n
i=1
1
x xi
(
)
Vn
hn
3. Since bandwidth is 2, Vn = hd = h1 = 2, the kernel function will
Problem 2
1. Build a decision tree:
The entropy equation is:
E=
P (wj ) log P (wj )
j
According to the data, we have the total labels 9+, 4-. So the entropy
for the whole data is:
9
9
4
4
log
log
= 0.8905
13
13 13
13
If we classify the data with Wakeup:
E
CSE 455/555 Spring 2011 Mid-Term Exam
Jason J. Corso
Computer Science and Engineering
SUNY at Buffalo
jcorso@buffalo.edu
Date 11 Mar 2011
Brevity is the soul of wit.
-Shakespeare
Directions Read Completely
The exam is closed book/notes. You have 50 minute
CSE 555 Spring 2010 Mid-Term Exam
Jason J. Corso
Computer Science and Engineering
SUNY at Buffalo
jcorso@buffalo.edu
Date 5 Mar 2010
The exam is worth 100 points total and each question is marked with its portion. The exam is
closed book/notes. You have 5
CSE 555 Spring 2009 Mid-Term Exam
Jason J. Corso
Computer Science and Engineering
University at Buffalo SUNY
jcorso@cse.buffalo.edu
Date 5 Mar 2009
The exam is worth 100 points total and each question is marked with its portion. The exam is closed book/no
Dynamic Bayesian Networks
Kevin P. Murphy www.ai.mit.edu/murphyk 12 November 2002
1 Introduction
Chapter ? introduced hidden Markov models (HMMs), and Chapter ? introduced state space models (SSMs), both of which are popular, but somewhat inexible, models
MSRI Workshop on Nonlinear Estimation and Classification, 2002.
The Boosting Approach to Machine Learning An Overview
Robert E. Schapire AT&T Labs ; Research Shannon Laboratory 180 Park Avenue, Room A203 Florham Park, NJ 07932 USA www.research.att.com/ sc
"
CSE 455/555 Homework 1 Random Forest on MNIST Digits Files
Jason Corso (jcorso@acm.org)
*This file is a hand-out for a homework assignment and leaves numerous places unfinished, intentionally.*
You are to use the prpy and examples code already provided
Introduction to Hidden Markov
Models
Slides Borrowed From Venu Govindaraju
Markov Models
Set of states:
Process moves from one state to another generating a
sequence of states :
Markov chain property: probability of each subsequent state
depends only o
IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No.2, pp. 228-233, 2001
PCA versus LDA
Aleix M. Mart nez and Avinash C. Kak Robot Vision Lab School of Electrical and Computer Engineering Purdue University, IN 47907-1285
faleix, ka
An Introduction to Locally Linear Embedding
Lawrence K. Saul AT&T Labs Research 180 Park Ave, Florham Park, NJ 07932 USA lsaul@research.att.com Sam T. Roweis Gatsby Computational Neuroscience Unit, UCL 17 Queen Square, London WC1N 3AR, UK roweis@gatsby.uc
Introduction to Hidden Markov Models
Slides Borrowed From Venu Govindaraju
Markov Models
Set of states: Process moves from one state to another generating a sequence of states : Markov chain property: probability of each subsequent state depends only on
Boosting and AdaBoost
Jason Corso
SUNY at Bualo
J. Corso (SUNY at Bualo)
Boosting and AdaBoost
1 / 62
Introduction
Weve talked loosely about
1
2
Lack of inherent superiority of any one particular classier; and
Some systematic ways for selecting a particul
CSE 455/555 Spring 2012 Homework 2
Jason J. Corso
TAs: Shujie Liu and Suxin Guo
Computer Science and Engineering
SUNY at Buffalo
jcorso@buffalo.edu
Date Assigned 20 March 2012
Date Due 19 April 2012
Homework must be submitted by midnight of the due-date,
CSE 455/555 Spring 2012 Homework 3
Jason J. Corso
TAs: Shujie Liu and Suxin Guo
Computer Science and Engineering
SUNY at Buffalo
jcorso@buffalo.edu
Date Assigned 10 April 2012
Date Due 26 April 2012
Homework must be submitted by midnight of the due-date,
Problem 1: No Free Lunch Theorem
1. For a particular algorithm h and a given training set D, the expected error
over all two-category problems can be represented as:
P (x)[1 (F (x), h(x)]P (F |D)
[E |D, h] =
F xD
/
where we suppose there are two categorie
Problem 1: Parametric Estimation
1. Since samples x1 , ., xn are drawn independently from the Bernoulli distribution,
n
p(D|)
(p(xi |)
=
i=1
n
xi (1 )(1xi )
=
i=1
n
i=1
=
xi
(1 )(n
n
i=1
xi )
Because xi cfw_0, 1, the previous equation can be expressed as
Problem 1: Bayesian Decision Rule
Solution:
1. a. p(x) can be calculated based on the prior and the likelihood distributions as:
p(x) = p(x| = 1)p( = 1) + p(x| = 2)p( = 2)
for
for
for
for
for
x < 0: p(x) = 0
0 x < 1: p(x) = 0.5 0.7 = 0.35
1 x 2: p(x) = 0.
CSE 455/555 Spring 2012 Homework 3
Jason J. Corso
TAs: Shujie Liu and Suxin Guo
Computer Science and Engineering
SUNY at Buffalo
jcorso@buffalo.edu
Date Assigned 10 April 2012
Date Due 26 April 2012
Homework must be submitted by midnight of the due-date,
A Tutorial on Dynamic Bayesian Networks
Kevin P. Murphy
MIT AI lab
12 November 2002
Modelling sequential data
Sequential data is everywhere, e.g.,
Sequence data (oine): Biosequence analysis,
text processing, .
Temporal data (online): Speech recognition
S1
S2
S3
ST
Y1
Y2
Y3
YT
W
X
Y
Z
e+(n)
p1
p2
p3
n
c1
c2
c3
e(n)
Y1
Y2
Y3
YT
(1)
(1)
S t+1
(2)
S t+1
(3)
S t-1
St
(2)
S t-1
St
(3)
(1)
(2)
(3)
S t-1
S t+1
Yt-1
X t-1
St
Yt
Yt+1
Xt
X t+1
(1)
S t+1
(2)
S t+1
(3)
(1)
St
(2)
St
S t-1
(3)
St
S t+1
Yt-1
Yt
Yt+1
S
ACCEPTED C ONFERENCE
ON
C OMPUTER V ISION
AND
PATTERN R ECOGNITION 2001
Rapid Object Detection using a Boosted Cascade of Simple Features
Paul Viola viola@merl.com Mitsubishi Electric Research Labs 201 Broadway, 8th FL Cambridge, MA 02139 Michael Jones mj
Structural Health Monitoring Using
Statistical Pattern Recognition
Supervised Learning Methods
Keith Worden and Graeme Manson
Presented by
Keith Worden
Los Alamos Dynamics Structural Dynamics and Mechanical Vibration Consultants
The Structural Health Moni