This preview shows pages 1–8. Sign up to view the full content.
1
Eric Xing
© Eric Xing @ CMU, 20062008
1
Machine Learning
Machine Learning
10
10
701/15
701/15
781, Fall 2008
781, Fall 2008
Hidden Markov Model
Hidden Markov Model
Eric Xing
Eric Xing
Lecture 16, November 3, 2008
Reading: Chap. 13, C.B book
Eric Xing
© Eric Xing @ CMU, 20062008
2
Dynamic clustering
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document 2
Eric Xing
© Eric Xing @ CMU, 20062008
3
Chromosomes of tumor cell:
Clustering Tumor Cell States
Eric Xing
© Eric Xing @ CMU, 20062008
4
Array CGH (comparative genomic
hybridization)
z
The basic assumption of a
CGH experiment is that the
ratio of the binding of test and
control DNA is proportional to
the ratio of the copy numbers
of sequences in the two
samples.
z
But various kinds of noises
make the true observations
less easy to interpret …
3
Eric Xing
© Eric Xing @ CMU, 20062008
5
Copy number profile for chromosome
1 from 600 MPE cell line
Copy number profile for chromosome
8 from COLO320 cell line
6070 fold amplification of CMYC region
Copy number profile for chromosome 8
in MDAMB231 cell line
deletion
DNA Copy number aberration
types in breast cancer
Eric Xing
© Eric Xing @ CMU, 20062008
6
A real CGH run
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document 4
Eric Xing
© Eric Xing @ CMU, 20062008
7
Out problem: how to cluster
sequential data?
Eric Xing
© Eric Xing @ CMU, 20062008
8
Hidden Markov Model:
from static to dynamic mixture models
Dynamic mixture
A
A
A
A
X
2
X
3
X
1
X
T
Y
2
Y
3
Y
1
Y
T
...
...
Static mixture
Static mixture
A
X
1
Y
1
N
5
Eric Xing
© Eric Xing @ CMU, 20062008
9
A
A
A
A
X
2
X
3
X
1
X
T
Y
2
Y
3
Y
1
Y
T
...
...
The sequence:
The underlying source:
Ploy NT,
genomic entities,
sequence of rolls,
dice,
Hidden Markov Models
Eric Xing
© Eric Xing @ CMU, 20062008
10
Example: The Dishonest Casino
A casino has two dice:
z
Fair die
P(1) = P(2) = P(3) = P(5) = P(6) = 1/6
z
Loaded die
P(1) = P(2) = P(3) = P(5) = 1/10
P(6) = 1/2
Casino player switches back&forth
between fair and loaded die once every
20 turns
Game:
1. You bet $1
2. You roll (always with a fair die)
3. Casino player rolls (maybe with fair die,
maybe with loaded die)
4. Highest number wins $2
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document 6
Eric Xing
© Eric Xing @ CMU, 20062008
11
Puzzles Regarding the Dishonest
Casino
GIVEN:
A sequence of rolls by the casino player
124552
6
4
6
214
6
14
6
13
6
13
666
1
66
4
66
1
6
3
66
1
6
3
66
1
6
3
6
1
6
515
6
1511514
6
1235
6
2344
QUESTION
z
How likely is this sequence, given our model of how the casino
works?
z
This is the
EVALUATION
problem in HMMs
z
What portion of the sequence was generated with the fair die, and
what portion with the loaded die?
z
This is the
DECODING
question in HMMs
z
How “loaded” is the loaded die? How “fair” is the fair die? How often
does the casino player change from fair to loaded, and back?
z
This is the
LEARNING
question in HMMs
Eric Xing
© Eric Xing @ CMU, 20062008
12
A Stochastic Generative Model
z
Observed sequence:
z
Hidden sequence (a parse or segmentation):
A
B
1
4
3
6
6
4
B
A
A
A
B
B
7
Eric Xing
© Eric Xing @ CMU, 20062008
13
Definition (of HMM)
z
Observation space
Alphabetic set:
Euclidean space:
z
Index set of hidden states
Index set of hidden states
z
Transition probabilities
between any two states
between any two states
or
z
Start probabilities
z
Emission probabilities
associated with each state
or in general:
A
A
A
A
x
2
x
3
x
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview. Sign up
to
access the rest of the document.
This note was uploaded on 01/26/2010 for the course MACHINE LE 10701 taught by Professor Ericp.xing during the Fall '08 term at Carnegie Mellon.
 Fall '08
 EricP.Xing

Click to edit the document details