Lecture-2.ppt

# Lecture-2.ppt - PART I Measurement of Motion Contents •...

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: PART I Measurement of Motion Contents • Image Motion Models • Optical Flow Methods – – – – – Horn & Schunck Lucas and Kanade Anandan et al Szeliski Mann & Picard • Video Mosaics 1 3-D Rigid Motion È X ¢˘ ÈX ˘ Èr11 Í˙ Í˙ Í ÍY ¢ ˙ = R ÍY ˙ + T = Ír21 ÍZ ¢ ˙ ÍZ ˙ Ír31 Î˚ Î˚ Î r12 r22 r32 r13 ˘ È X ˘ ÈTX ˘ ˙Í ˙ Í ˙ r23 ˙ ÍY ˙ + ÍTY ˙ r33 ˙ Í Z ˙ ÍTZ ˙ ˚Î ˚ Î ˚ Translation (3 unknowns) Rotation matrix (9 unknowns) Rotation X = R cos f Y = R sin f Y ( X ¢, Y ¢, Z ¢) X ¢ = R cos(Q + f ) = R cos Q cos f - R sin Q sin f Y ¢ = R sin(Q + f ) = R sin Q cos f + R cos Q sin f X ¢ = X cos Q - Y sin Q Y ¢ = X sin Q + Y cos Q R Y’ R q ( X ,Y , Z ) Y f X X’ X Z È X ¢˘ Ècos Q - sin Q 0˘ È X ˘ ÍY ¢ ˙ = Í sin Q cos Q 0˙ ÍY ˙ Í˙Í ˙Í ˙ ÍZ ¢ ˙ Í 0 0 1˙ Í Z ˙ Î˚Î ˚Î ˚ 2 Rotation (continued) Y È1 0 0˘ Í ˙ R = Í0 1 0˙ Í0 0 1˙ Î ˚ v W X u Z Ècos Q - sin Q 0˘ Í ˙ R = Í sin Q cos Q 0˙ Í0 0 1˙ Î ˚ Y v’ u’ Q Z Ècos b 0 - sin b ˘ Í ˙ R=Í 0 1 0˙ Í sin b 0 cos b ˙ Î ˚ W W’ X Q Y v b X b Z u’ Euler Angles Ècos a cos b cos a sin b sin g - sin a cos g cos a sin b cos g + sin a sin g ˘ a g R = RZ RYb RX = Í sin a cos b sin a sin b sin g + cos a cos g sin a sin b cos g - cos a sin g ˙ Í ˙ Í - sin b ˙ cos b sin g cos b cos g Î ˚ if angles are small( cos Q ª 1 È1 R=Í a Í Í- b Î -a 1 g sin Q ª Q b˘ -g ˙ ˙ 1˙ ˚ 3 Image Motion Models Displacement Model 4 Orthographic Projection (X,Y,Z) World point Image Plane y image y =Y x=X Orthographic Projection È X ¢˘ Í˙ ÍY ¢ ˙ = ÍZ ¢ ˙ Î˚ ÈX ˘ Èr11 r12 Í˙ Í R ÍY ˙ + T = Ír21 r22 ÍZ ˙ Ír31 r32 Î˚ Î x= X y=Y x ¢ = r11 x + r12 y + (r13 Z + TX ) y ¢ = r21 x + r22 y + (r23 Z + TY ) x¢ = a1 x + a2 y + b1 y¢ = a3 x + a4 y + b2 x ¢ = Ax + b Èa A=Í 1 Îa3 r13 ˘ È X ˘ ÈTX ˘ ˙Í ˙ Í ˙ r23 ˙ ÍY ˙ + ÍTY ˙ r33 ˙ Í Z ˙ ÍTZ ˙ ˚Î ˚ Î ˚ (x,y)=image coordinates, (X,Y,Z)=world coordinates Affine Transformation a2 ˘ Èb1 ˘ ˙, b = Íb ˙ a4 ˚ Î 2˚ 5 Orthographic Projection (contd.) È X ¢˘ È X ˘ È 1 - a b ˘ È X ˘ ÈTX ˘ ÍY ˙ = R ÍY ˙ + T = Í a 1 - g ˙ ÍY ˙ + ÍT ˙ Í˙ Í˙ Í ˙Í ˙ Í Y ˙ ÍZ ¢ ˙ ÍZ ˙ Í- b g 1 ˙ ÍZ ˙ ÍTZ ˙ Î˚ Î˚ Î ˚Î ˚ Î ˚ x¢ = x - ay + bZ + TX y¢ = ax + y - gZ + TY Perspective Projection (X,Y,Z) World point Image Plane f Lens y Z image -y f = Y Z fY y=Z x=- fX Z 6 Perspective Projection È X ¢˘ ÈX ˘ Èr11 Í˙ Í˙ Í ÍY ¢ ˙ = R ÍY ˙ + T = Ír21 ÍZ ¢ ˙ ÍZ ˙ Ír31 Î˚ Î˚ Î r12 r22 r32 X ¢ = r11 X + r12Y + r13 Z + TX Y ¢ = r21 X + r22Y + r23 Z + TY Z ¢ = r31 X + r32Y + r33 Z + TZ x¢ = X¢ Z¢ y¢ = Y¢ Z¢ focal length = -1 r13 ˘ È X ˘ ÈTX ˘ ˙Í ˙ Í ˙ r23 ˙ ÍY ˙ + ÍTY ˙ r33 ˙ Í Z ˙ ÍTZ ˙ ˚Î ˚ Î ˚ TX Z x¢ = TZ r31 x + r32 y + r33 + Z TY r21 x + r22 y + r23 + Z y¢ = TZ r31 x + r32 y + r33 + Z r11 x + r12 y + r13 + scale ambiguity Plane+Perspective(projective) aX + bY + cZ = 1 [a equation of a plane b È X ¢˘ Í˙ ÍY ¢ ˙ = ÍZ ¢ ˙ Î˚ ÈX ˘ Í˙ c ÍY ˙ = 1 ÍZ ˙ Î˚ ÈX ˘ Í˙ RÍ Y ˙ + T ÍZ ˙ Î˚ È X ¢˘ È X ˘ ÈX ˘ Í ˙ Í˙ Í˙ ÍY ¢ ˙ = R Í Y ˙ + T a b c ÍY ˙ ÍZ ¢ ˙ Í Z ˙ ÍZ ˙ Î ˚ Î˚ Î˚ [ È X ¢˘ È X ˘ Í ˙ Í˙ ÍY ¢ ˙ = AÍY ˙ ÍZ ¢ ˙ ÍZ ˙ Î ˚ Î˚ [ A= R+T a b c 3d rigid motion 7 Plane+Perspective(projective) È X ¢˘ È X ˘ Í ˙ Í˙ ÍY ¢ ˙ = AÍY ˙ ÍZ ¢ ˙ ÍZ ˙ Î ˚ Î˚ x¢ = x¢ = X ¢ = a1 X + a2Y + a3 Z Y ¢ = a4 X + a5Y + a6 Z Z ¢ = a7 X + a8Y + a9 Z X¢ Z¢ a1 X + a2Y + a3 Z a7 X + a8Y + a9 Z y¢ = a4 X + a5Y + a6 Z a7 X + a8Y + a9 Z x¢ = a1 x + a2 y + a3 a7 x + a8 y + a9 y¢ = a4 x + a5 y + a6 scale ambiguity a7 x + a8 y + a9 y¢ = Y¢ Z¢ focal length = -1 a9 = 1 Plane+perspective (contd.) x¢ = a1 x + a2 y + a3 a7 x + a8 y + 1 a x + a5 y + a6 y¢ = 4 a7 x + a8 y + 1 a2 ˘ Èa È x¢ ˘ X¢ = Í ˙, A = Í 1 , a4 a5 ˙ y ¢˚ Î Î ˚ Èa3 ˘ È a7 ˘ Èx ˘ b = Í ˙, C = Í ˙, X = Í ˙ Î y˚ Î a6 ˚ Îa8 ˚ X¢ = AX + b CT X +1 Projective 8 Least Squares • Eq of a line È x1 Í ÍM Í Í Í Íx n Î mx + c = y • Consider n points † mx1 + c = y1 M mx n + c = y n È y1 ˘ 1˘ ˙ Í˙ M˙ M Èm˘ Í ˙ ˙Í ˙ = Í ˙ ˙Îc ˚ Í ˙ ˙ Í˙ Íy n ˙ 1˙ ˚ Î˚ Ap = Y † † † Least Squares Fit Ap = Y A T Ap = A TY p = (A T A )-1 A TY n † min Â ( y i - mx i - c ) 2 i=1 † 9 Projective • If point correspondences (x,y)<-->(x’,y’) are known • a’s can be determined by least squares fit x¢ = a1 x + a2 y + a3 a7 x + a8 y + 1 y¢ = a4 x + a5 y + a6 a7 x + a8 y + 1 Èa1 ˘ Í˙ Ía2 ˙ È ˘Ía3 ˙ È M ˘ M Íx i y i 1 0 0 0 - x i x ¢ - y i x ¢˙Ía4 ˙ Í x ¢˙ i iÍ ˙ i =Í ˙ Í ˙ i i Í0 0 0 x i y i 1 - x i y ¢ - y i y i¢˙Ía5 ˙ Í y ¢˙ M Î ˚Ía6 ˙ Î M ˚ Í˙ Ía7 ˙ Ía ˙ Î 8˚ † Affine •If point correspondences (x,y)<-->(x’,y’) are known • a’s and b’s can be determined by least squares fit x¢ = a1 x + a2 y + b1 y¢ = a3 x + a4 y + b2 È M Íx i y i 1 0 0 0 Í Í0 0 0 x i y i 1 Í M Î Èa1 ˘ Í˙ Ía2 ˙ ˘Íb1 ˙ È M ˘ ˙Ía ˙ Í x ¢˙ ˙Í 3 ˙ = Í i˙ ˙Ía4 ˙ Í y ¢˙ i ˙ ˚Íb2 ˙ Î M ˚ Í˙ Í˙ Í˙ Î˚ † 10 Summary of Displacement Models Translation x¢ = x + b1 x ¢ = a1 + a2 x + a3 y + a4 x 2 + a5 y 2 + a6 xy y¢ = y + b2 y ¢ = a7 + a8 x + a9 y + a10 x 2 + a11 y 2 a12 xy Rigid x¢ = x cosq - y sin q + b1 y¢ = x sin q + y cosq + b2 x¢ = a1 x + a2 y + b1 y¢ = a3 x + a4 y + b2 Affine a x + a2 y + b1 x¢ = 1 c1 x + c2 y + 1 Projective y¢ = a3 x + a4 y + b1 c1 x + c2 y + 1 Biquadratic x¢ = a1 + a2 x + a3 y + a4 xy y¢ = a5 + a6 x + a7 y + a8 xy Bilinear x ¢ = a1 + a2 x + a3 y + a4 x 2 + a5 xy y ¢ = a6 + a7 x + a8 y + a4 xy + a5 y 2 Pseudo Perspective Displacement Models (contd) • Translation – simple – used in block matching – no zoom, no rotation, no pan and tilt • Rigid – rotation and translation – no zoom, no pan and tilt 11 Displacement Models (contd) • Affine – rotation about optical axis only – can not capture pan and tilt – orthographic projection • Projective – exact eight parameters (3 rotations, 3 translations and 2 scalings) – difficult to estimate Displacement Models (contd) • Biquadratic – obtained by second order Taylor series – 12 parameters • Bilinear – obtained from biquadratic model by removing square terms – most widely used – not related to any physical 3D motion • Pseudo-perspective – obtained by removing two square terms and constraining four remaining to 2 degrees of freedom 12 Spatial Transformations translation rotation shear rigid affine Decomposition of Affine A = SVD = S ( DD -1 )VD = ( SD)( D -1VD) Ècos a = R(a )C = Í Î sin a È1 A = DÍ r Í Îa - sin a ˘ È v1 Í cos a ˙ Îv h ˚ vh ˘ v2 ˙ ˚ ˘ 0˙Ècos b - sin b ˘ Í ˙ ˙Îsin b cos b ˚ r˚ D = scale _ factor = sx sy , r = scale _ ratio = sx , a = skew sy † † 13 Displacement Models (contd) Affine Mosaic 14 Projective Mosaic Instantaneous Velocity Model 15 3-D Rigid Motion È X ¢˘ È 1 - a b ˘ È X ˘ ÈTX ˘ ÍY ¢ ˙ = Í a 1 - g ˙ ÍY ˙ + ÍT ˙ Í˙Í ˙Í ˙ Í Y ˙ ÍZ ¢ ˙ Í- b g 1 ˙ ÍZ ˙ ÍTZ ˙ Î˚Î ˚Î ˚ Î ˚ È X ¢˘ Ê È 0 ÍY ¢ ˙ = Á Í a Í ˙ ÁÍ Í Z ¢ ˙ Á Í- b Î ˚ ËÎ -a 0 g È X ¢ - X ˘ È 0 - a b ˘ È X ˘ ÈTX ˘ ÍY ¢ - Y ˙ = Í a 0 - g ˙ ÍY ˙ + ÍTY ˙ Í ˙Í ˙Í ˙ Í ˙ Í Z ¢ - Z ˙ Í- b g 0 ˙ Í Z ˙ ÍTZ ˙ Î ˚Î ˚Î ˚ Î ˚ ˙ ÈX ˘ È 0 -W3 W2 ˘ÈX ˘ ÈV1 ˘ Í˙Í ˙Í ˙ Í ˙ ˙ Y ˙ = Í W3 0 -W1˙ÍY ˙ + ÍV2 ˙ Í ÍZ ˙ Í-W 0 ˙ÍZ ˙ ÍV3 ˙ ˚Î ˚ Î ˚ Î ˙ ˚ Î 2 W1 b ˘ È1 0 0˘ ˆ†X ˘ ÈTX ˘ È ˜ - g ˙ + Í0 1 0˙ ˜ ÍY ˙ + ÍTY ˙ ˙Í ˙Í ˙ Í ˙ 0 ˙ Í0 0 1˙ ˜ Í Z ˙ ÍTZ ˙ ˚Î ˚ ¯Î ˚ Î ˚ 3-D Rigid Motion ˙ X = W2 Z - W3Y + V1 ˙ Y = W3 X - W1Z + V2 ˙ Z = W1Y - W2 X + V3 ˙ X = W¥ X+ V † ÈX ˘ X = ÍY ˙, Í˙ ÍZ ˙ Î˚ ÈW1 ˘ W = ÍW 2 ˙ Í˙ ÍW 3 ˙ Î˚ Cross Product † 16 Orthographic Projection ˙ X = W2 Z - W3Y + V1 ˙ Y = W3 X - W1Z + V2 y =Y x=X ˙ Z = W1Y - W2 X + V3 ˙ u = x = W2 Z - W3 y + V1 ˙ v = y = W3 x - W1Z + V2 (u,v) is optical flow † Perspective Projection (arbitrary flow) fX Z fY y= Z x= ˙ ˙ fZX - fXZ ˙ ˙ X Z -x Z2 Z Z ˙ - fYZ ˙ ˙ ˙ fZY Y Z ˙ v=y= = f -y 2 Z Z Z ˙ u= x= =f ˙ X = W2 Z - W3Y + V1 ˙ Y = W3 X - W1Z + V2 ˙ † Z = W1Y - W2 X + V3 V V1 W W + W 2 ) - 3 x - W 3 y - 1 xy + 2 x 2 Z Z f f V V W W v = f ( 2 - W1 ) + W 3 x - 3 y + 2 xy - 1 y 2 Z Z f f u= f( † 17 Plane+orthographic(Affine) Z = a + bX + cY b1 = V1 + aW 2 u = V1 + W 2 Z - W 3 y a1 = bW 2 v = V2 + W 3 x - W1Z a2 = cW 2 - W 3 u = b1 + a1 x + a2 y b2 = V2 - aW1 v = b2 + a3 x + a4 y a3 = W 3 - bW1 a4 = -cW1 u = Ax + b Plane+Perspective (pseudo perspective) V V W W u = f ( 1 + W 2 ) - 3 x - W 3 y - 1 xy + 2 x 2 Z Z f f V V W W v = f ( 2 - W1 ) + W 3 x - 3 y + 2 xy - 1 y 2 Z Z f f Z = a + bX + cY 11b c = - x- y Zaa a u = a1 + a2 x + a3 y + a4 x 2 + a5 xy v = a6 + a7 x + a8 y + a4 xy + a5 y 2 18 ...
View Full Document

## This note was uploaded on 06/12/2011 for the course CAP 6411 taught by Professor Shah during the Spring '09 term at University of Central Florida.

Ask a homework question - tutors are online