EM+Algorithm - Xiao Han,2008 EM Algorithm Xiao Han...

Info iconThis preview shows pages 1–12. Sign up to view the full content.

View Full Document Right Arrow Icon
EM Algorithm Xiao Han [email protected] Ver.1.2008.10.19 Reference: Christopher M.Bishop, Pattern Recognition and Machine Learning 2008/10/19 Xiao Han,2008 Xiao Han
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
预备知识 概率(加法、乘法、条件概率、 i.i.d. 、多维随机 变量、高斯分布、贝叶斯、 Maximum log- likelihood 求导(偏导、向量求导、矩阵求导、拉格朗日乘 法) 2008/10/19 2 Xiao Han
Background image of page 2
问题的来源 给定一些观察数据 x, 假设 x 符合如下的混合高斯分 我们要求混合高斯分布的三组参数 2008/10/19 3 Xiao Han
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
问题图示 2008/10/19 4 Xiao Han
Background image of page 4
简化的问题 该混合高斯分布一共有 k 个分布,并且 对于每一个 观察到的 x ,如果我们同时还知道它是属于 k 中哪 一个分布的 ,则求各个参数并不是件难事 比如用 z 来表示每一个高斯分布,那么我们的观察 集不仅仅是 {x 1, x 2, x 3… }, 而是 {(x 1, z 2 ),(x 2, z 3 ), (x 3, z 1 ) } 2008/10/19 5 Xiao Han
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
简化问题的图示 2008/10/19 6 Xiao Han
Background image of page 6
实际问题 而现实往往是: 我们不知道每个 x 属于哪个分布,也就是说 z 是我 们观察不到的, z 是隐藏变量 (latent variable) 2008/10/19 7 Xiao Han
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
引入两个概率 p(x) p(x,z) 2008/10/19 8 Xiao Han
Background image of page 8
隐藏变量 Z 为了将 k 个高斯分布用一个随机变量表示 可以采用 1-of-K 的表示法,例如 k=3 时: z 1 =1 表示 (1 0 0), p(z 1 =1)=π 1, z 2 =1 表示 (0 1 0), p(z 2 =1)=π 2, z 3 =1 表示 (0 0 1), p(z 3 =1)=π 3 于是 这里的粗体 z 表示的是形如 (1 0 0) 这样的向量 2008/10/19 9 Xiao Han
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
隐藏变量与混合高斯分布 Z 引入后 最终得到 2008/10/19 10 Xiao Han
Background image of page 10
约定 在后面的讨论中我们约定 P(x) P(x| 参数 ) 是相同 的。例如混合高斯分布中 P(x)=P(x|π,μ,Σ) 另外,对于观察集 {X} 中的各个观察值 x i , 我们认为 相互之间独立。特别的,如果 x 1 x 5 x 9 来自于同 一高斯分布,我们认为他们满足 i.i.d.( 独立同分布 ) 2008/10/19 11 Xiao Han
Background image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 12
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 03/14/2011 for the course MATH 101 taught by Professor No during the Spring '11 term at Nanjing University.

Page1 / 35

EM+Algorithm - Xiao Han,2008 EM Algorithm Xiao Han...

This preview shows document pages 1 - 12. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online