lect1014h - Visual Lipreading Image Sequences of A to J 1...

Info iconThis preview shows pages 1–5. Sign up to view the full content.

View Full Document Right Arrow Icon
1 Visual Lipreading Image Sequences of “A” to “J”
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
2 Particulars • Problem : Pattern differ spatially • Solution : Spatial registration using SSD • Problem : Articulations vary in length, and thus, in number of frames. • Solution : Dynamic programming for temporal warping of sequences. • Problem : Features should have compact representation. • Solution : Principle Component Analysis. Feature Subspace Generation • Generate a lower dimension subspace onto which image sequences are projected to produce a vector of coefficients. • Components – Sample Matrix – Most Expressive Features
Background image of page 2
3 Generating the Sample Matrix • Consider letters, each of which has a training set of K sequences. Each sequence is compose of images: P I I I , , , 2 1 K • Collect all gray-level pixels from all images in a sequence into a vector: )) , ( , ), 1 , 1 ( ), , ( , ), 1 , 1 ( ), , ( , ), 1 , 1 ( ( 2 2 1 1 N M I I N M I I N M I I u P P K K K K = e . Generating the Sample Matrix For letter , collect vectors into matrix T [ ] K u u u T K , , 2 1 = w Create sample matrix A : [ ] e T T T A K , , 2 1 = w •The eigenvectors of a matrix are defined as: T AA L = i i i L f l f =
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
4 The Most Expressive Features Use Q most significant eigenvectors. •Any image sequence, u, can be represented as: a a u Q n n n f f = = = 1 The linear coefficients can be computed as : n T n u a f = is an orthonormal basis of the sample matrix. f Training Process • Model Generation – Warp all the training sequences to a fixed length. – Perform spatial registration (SSD).
Background image of page 4
Image of page 5
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 19

lect1014h - Visual Lipreading Image Sequences of A to J 1...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online