Space_Time_Tracking_Presentation(1) - Space Time Tracking...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Space Time Tracking ECCV 2002 ECCV Lorenzo Torresani Christoph Bregler Outline Outline Problem Background Structure from Motion Matrix Decomposition Non-Rigid Motion Estimation Non-Rigid Shapes Estimation Results Problem Problem “To track feature points on non-rigid objects To without using any prior model” without Rank of a Matrix Rank A = a1 a 2 .. .. .. .. aN M N [ Rank (A) = Number of linearly independent vectors in a1 b1 b 2 B = . . . . bM . . . . a3 a N ] M N Rank (B) = Number of linearly independent vectors in For M x N the Rank of A ≤ min (M,N) a2 [ b1 b2 b3 bM ] Rank of a Matrix cont’d cont’d C = AMxN TNxP Rank C = ? Columns of C are Linear combination of Columns of A There are only N independent vectors in A Rank C = N SVD SVD SVD for a matrix A writes A as a product of three matrices: •U •D •V • Every m x n matrix has a singular value decomposition D nxn A mxn U mxn VT nxn U,V have orthogonal columns Tomasi Kanade Structure from Motion Given N 2D trajectories taken over F images, recover 3D Given structure and motion (Camera pose) structure Frame 1 Frame 2 ………… Frame F • Assumption: • 3D Object is rigid • Orthographic Projection • Tracks can be computed using any standard tracker (KLT etc) Tomasi Kanade Structure from Motion cont’d Tomasi Assume a set of P 3D points on a rigid object (structure) S = [P1, P2 …….. PP ] Orthographic Projection u p = = M 2 x 3 ( R3 x 3 P + T3 x1 ) v where (u,v) are image coordinates and M is orthographic projection where matrix matrix Subtract mean of all u’s and v’s to center the world coordinate frame at the Subtract center of the object. center This will get rid of T in the above equation u p = = M 2 x 3 ( R3 x 3 P ) v Tomasi Kanade Structure from Motion cont’d 2D 2D coordinates of N points over F images can be written in one matrix written W is called the measurement/tracking matrix Rank 3 U W = = R2 Fx3 S 3 xN V From W to R and S From Force the rank of W to be 3 SVD Steps Steps Matrix Decomposition of W matrix for nonrigid objects Estimate Estimate Motion Matrix using reliable set of points of Estimate Estimate shape basis (S) for all other feature points (unreliable) feature For Non Rigid Constraint For u p = = M 2 x 3 ( R3 x 3 P + T3 x1 ) v u p = = M 2 x 3 ( R3 x 3 S Non − Rigid + T3 x1 ) v 3D Non Rigid Shape Model Linear Combination of K Basis Shapes Each basis shape is Si 3 x P matrix describing P points S S1 S2 = l1 S 1 + l2 S 2 ……………… Courtesy Christopher Bregler +…+ lKSK SK Matrix Decomposition Matrix Project P points of shape S Scaled Orthographic Projection Move world coordinate to object centeroid (This will get Move rid of T) rid W Q M - Tracking Matrix Tracking 2F x 3K Complete 2D Tracks or Flow Rank of W 3K In Tomasi Kanade it was 3 3K x P Non Rigid Motion Estimation Non Since W is rank-deficient, Q can be estimated w/o Since is the full availability of W r <= 3K point tracks will span the space of the trajectories of all the points (as rank of W is r) trajectories known reliable tracks ? W ? = Q’ M’ Courtesy Christopher Bregler r=9 Trajectory Constraint Trajectory t=F t=2 t=1 . .frames .. wi : full trajectory . = Q ’ . . 3D positions of point i for the K modes of deformation mi • Generate m trajectories (hypothesis) using Factored Sampling • Evaluate w by computing sum of square difference around point i. Courtesy Christopher Bregler • Each column mi of unreliable M is computed as expected value of posterior. Results Results ...
View Full Document

This note was uploaded on 06/13/2011 for the course CAP 6412 taught by Professor Staff during the Spring '08 term at University of Central Florida.

Ask a homework question - tutors are online