Statistics 24610/37500, Spring 2011
Homework 1 (due in class on Thursday, 7 April 2011)
Note: Problems 1 and 2 are to be done individually. Problem 3 has to be done in groups of 4-5 students.
Turn in a group report for your work on problem 3. Keep the cod
Statistics 24610/37500, Spring 2011
Homework 6 (due in class on Tuesday, 31 May 2011)
Note: All problems are to be done individually. Keep the code you develop.
1. Let p = P (Z > 4.5) for a standard normal r.v. Z N (0, 1).
(a) Explain how you would comput
Statistics 24610/37500, Spring 2011
Homework 5 (due in class on Tuesday, 24 May 2011)
Problem 1 has to be done individually. Problems 2 and 3 can, but dont have to be done in groups of 2
students. You have to write your own computer code for these problem
Linear Algebra Review
STAT 343, Fall 2011
October 10, 2011
Linear maps
Denition
A function f : Rm ! Rn is linear if
f (x + y ) = f (x ) + f (y );
f (ax ) = af (x ), 8a 2 R.
Denition
A basis for a vector space V is a minimal set of vectors B = cfw_vi ,
i
Stat 246/375
Mixture Model EM Algorithm :
Start: Initialize ,
E-step: compute " ( zik ) =
Spring 2011
19 April 2011
# k p( X k | $ k )
k
% p( X
i
&i, k
|$l )
l =1
!
new
M-step: compute " k =
n
nk
, where n k = #" ( zik )
n
i =1
" knew obta
Stat 246/375 Pattern Recognition
Professor Mathias Drton
Spring 2011
Lecture 1
29 March, 2011
Stochastic Gradient Descent
Stochastic gradient descent is an optimization method for minimizing an objective
function that is written as a sum of differentiable
Spring 2011, STAT 24610/37500
Lecture Notes (April 12th, 2011)
4. Linear Models for Classication
(a) Fishers Discriminant Analysis (continued)
T B
() is given by eigenvector to largest eigenvalue of 1 B .
Lemma: Solution to max
p
R T
Proof:
If is an iden
Statistics 24610/37500, Spring 2011
Homework 2 (due in class on Thursday, 14 April 2011)
1. Let X = (X1 , X2 ) be bivariate normal with mean zero and covariance matrix = (ij ).
(a) Write X2 = X1 + Z where Cov[Z, X1 ] = 0. Provide an explicit formula for a
Statistics 24610/37500, Spring 2011
Homework 4 (due in class on Thursday, 12 May 2011)
Note: Problems 1, 2 and 3 are to be done individually. Problem 4 is a special assignment; pay attention to
the instructions given there. Keep the computer code you deve
STAT246/375 - Lecture 3 Notes
Asymptotic risk of nearest neighbor classication
(X, Y ) p(x, y ), X Rd Test Case
RB = E[L(Y, fB (X )] Bayes Risk
(n)
fN N (x): nearest-neighbor classier based on training data
iid
(X (1) , Y (1) ), . . . , (X (n) , Y (n) ),
Stat 246/375
Lecture Notes
April 21, 2011
6 EM Algorithm in General
X: observed variables
Z: unobserved/ hidden variables/ latent variables
So the joint distribution of X, Z is p(x, z |), the marginal distribution for X is
p(x|)
Our goal is to maximize in
Spring 2011, STAT 24610/37500
Lecture Notes (April 14th, 2011)
4. Linear Models for Classication (Continued)
(d) Discriminative Model
Consider the linear regression model:
P (Y = y |X = x) = E 1cfw_Y =y |X = x
where the response is a 0-1 valued vector, an
Stat 246/375 Lecture Notes April 28th
Factorization in undirected graphical models. Let G be an undirected graph on the set of
nodes [m] = cfw_1, 2, . . . , m with edge set E . A clique C [m] in the graph is a collection of nodes
such that (i, j ) E for e
Stat 246/375: Pattern Matching
Lecture 2
Professor Mathias Drton
Spring 2011
31 March 2011
Classication
We wish to minimize
y
L(y, y )p(y |x). In practice, p(x, y ) is unknown, so we must estimate it.
Suppose we have some set of training data of the form
Statistics 24610/37500, Spring 2011
Homework 3 (due in class on Thursday, 28 April 2011)
Note: Problems 1, 2 and 3 are to be done individually. Problem 4 can, but doesnt have to be done in groups
of 2 students. Keep the code you develop as you may be aske