Unformatted text preview: Space Time Tracking
ECCV 2002
ECCV Lorenzo Torresani
Christoph Bregler Outline
Outline Problem Background Structure from Motion Matrix Decomposition NonRigid Motion Estimation NonRigid Shapes Estimation Results Problem
Problem
“To track feature points on nonrigid objects
To
without using any prior model”
without Rank of a Matrix
Rank A = a1 a 2 ..
.. ..
.. aN M N [ Rank (A) = Number of linearly independent vectors in a1 b1 b
2 B = .
. . . bM .
. .
. a3 a N ] M N
Rank (B) = Number of linearly independent vectors in
For M x N the Rank of A ≤ min (M,N) a2 [ b1 b2 b3 bM ] Rank of a Matrix cont’d
cont’d C = AMxN TNxP
Rank C = ? Columns of C are Linear combination of Columns of A There are only N independent vectors in A Rank C = N SVD
SVD
SVD for a matrix A writes A as a product of three
matrices:
•U
•D
•V
• Every m x n matrix has a singular value decomposition D
nxn
A
mxn U
mxn VT
nxn
U,V have orthogonal
columns Tomasi Kanade Structure from Motion Given N 2D trajectories taken over F images, recover 3D
Given
structure and motion (Camera pose)
structure Frame 1 Frame 2 ………… Frame F • Assumption:
• 3D Object is rigid
• Orthographic Projection
• Tracks can be computed using any standard
tracker (KLT etc) Tomasi Kanade Structure from Motion cont’d
Tomasi Assume a set of P 3D points on a rigid object (structure)
S = [P1, P2 …….. PP ] Orthographic Projection u p = = M 2 x 3 ( R3 x 3 P + T3 x1 )
v where (u,v) are image coordinates and M is orthographic projection
where
matrix
matrix Subtract mean of all u’s and v’s to center the world coordinate frame at the
Subtract
center of the object.
center This will get rid of T in the above equation u p = = M 2 x 3 ( R3 x 3 P )
v Tomasi Kanade Structure from Motion cont’d 2D
2D coordinates of N points over F images can be
written in one matrix
written W is called the measurement/tracking matrix Rank 3 U W = = R2 Fx3 S 3 xN
V From W to R and S
From Force the rank of W to be 3 SVD Steps
Steps Matrix Decomposition of W matrix for nonrigid objects Estimate
Estimate Motion Matrix using reliable set of points
of Estimate
Estimate shape basis (S) for all other
feature points (unreliable)
feature For Non Rigid Constraint
For
u p = = M 2 x 3 ( R3 x 3 P + T3 x1 )
v u p = = M 2 x 3 ( R3 x 3 S Non − Rigid + T3 x1 )
v 3D Non Rigid Shape Model Linear Combination of K Basis Shapes
Each basis shape is Si 3 x P matrix describing P points S S1 S2 = l1 S 1 + l2 S 2 ……………… Courtesy Christopher Bregler +…+ lKSK SK Matrix Decomposition
Matrix Project P points of shape S
Scaled Orthographic Projection Move world coordinate to object centeroid (This will get
Move
rid of T)
rid W Q M  Tracking Matrix
Tracking 2F x 3K
Complete 2D Tracks or Flow Rank of W 3K
In Tomasi Kanade it was 3 3K x P Non Rigid Motion Estimation
Non
Since W is rankdeficient, Q can be estimated w/o
Since
is
the full availability of W r <= 3K point tracks will span the space of the
trajectories of all the points (as rank of W is r)
trajectories known
reliable tracks ?
W ? =
Q’ M’ Courtesy Christopher Bregler r=9 Trajectory Constraint
Trajectory
t=F
t=2
t=1 . .frames
..
wi : full trajectory . = Q ’ .
. 3D positions of point i
for the K modes of
deformation mi • Generate m trajectories (hypothesis) using Factored
Sampling
• Evaluate w by computing sum of square difference around
point i.
Courtesy Christopher Bregler • Each column mi of unreliable M is computed as expected
value of posterior. Results
Results ...
View
Full
Document
This note was uploaded on 06/13/2011 for the course CAP 6412 taught by Professor Staff during the Spring '08 term at University of Central Florida.
 Spring '08
 Staff

Click to edit the document details