Lecture-7.ppt - Lecture-7 Feature-based Registration Steps...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Lecture-7 Feature-based Registration Steps in Feature-based Registration • Find features • Establish correspondence (correlation, point correspondence) • Fit transformation • Apply transformation (warp) 1 Features • • • • Any pixels Corner points Interest points Features obtained using Gabor/Wavelet filters • Straight lines • Line intersections Transformations • • • • Affine Projective Psuedo-perspective Rational polynomial 2 Good Features to Track • Corner like features • Moravec’s Interest Operator Corner like features ÈÂf2 ÍQ x C =Í Í f x f y ÎQ fy ˘ ˙ Q  f y2 ˙ ˙ Q ˚ Âf x Èl1 0 ˘ C=͆ 0 l2 ˙ Î ˚ 3 Corners • For • If perfectly uniform region 1 = l2 = 0 l Q contains an ideal step edge, then l2 = 0, l1 > 0 • if Q contains a corner of black square on white background l1 ≥ l2 f 0 Algorithm Corners • Compute the image gradient over entire image f. • For each image point p: – form the matrix C over (2N+1)X(2N+1) neighborhood Q of p; – compute the smallest eigenvalue of C; – if eigenvalue is above some threshold, save the coordinates of p into a list L. 4 Algorithm Corners • Sort L in decreasing order of eigenvalues. • Scanning the sorted list top to bottom: for each current point, p, delete all other points on the list which belong to the neighborhood of p. Results 5 Results 6 Moravec’s Interest Operator Algorithm • Compute four directional variances in horizontal, vertical, diagonal and antidiagonal directions for each 4 by 4 window. • If the minimum of four directional variances is a local maximum in a 12 by 12 overlapping neighborhood, then that widow (point) is interesting. 7 8 3 2 Vh =   ( P( x + i, y + j ) - P( x + i + 1, y + j )) 2 j =0 i =0 2 3 Vv =   ( P( x + i, y + j ) - P( x + i, y + j + 1)) 2 j =0 i =0 2 2 Vd =   ( P( x + i, y + j ) - P( x + i + 1, y + j + 1)) 2 j =0 i =0 2 3 Va =   ( P( x + i, y + j ) - P( x + i - 1, y + j + 1)) 2 j = 0 i =1 V ( x, y ) = min(Vh ( x, y ),Vv ( x, y ),Vd ( x, y ),Va ( x, y )) Ï1 ifV ( x, y )local max I ( x, y ) = Ì 0 0therwise Ó 9 Correlation • Similarity/Dissimilarity Measures – – – – Sum of Squares Difference (SSD) Normalized Correlation Mutual Correlation Mutual information H(X;Y)=H(X)-H(X|Y) • Use – Gray levels – Laplacian of Gaussian – Gradient magnitude 10 Block Matching Frame k-1 Frame k 8X8 16X16 Block Matching • For each 8X8 block, centered around pixel (x,y) in frame k, Bk – Obtain 16X16 block in frame k-1, centered around (x,y), Bk-1 – Compute Sum of Squares Differences (SSD) between 8X8 block, Bk, and all possible 8X8 blocks in Bk-1 – The 8X8 block in Bk-1 centered around (x’,y’), which gives the least SSD is the match – The displacement vector (optical flow) is given by u=xx’; v=y-y’ 11 Sum of Squares Differences (SSD) -7 -7 2 (u ( x, y ), v( x, y )) = arg min u ,v = -3K3   ( f k ( x + i, y + j ) - f k -1 ( x + i + u , y + j + v) ) i =0 j =0 Minimum Absolute Difference (MAD) -7 -7 (u( x, y ), v ( x, y )) = arg min u,v = -3K 3   | ( f k ( x + i, y + j ) - f k -1 ( x + i + u, y + j + v )) | i= 0 j = 0 † 12 Maximum Matching Pixel Count (MPC) Ï1 if | f k ( x, y ) - f k -1( x + u, y + v ) |£ t T ( x, y; u, v ) = Ì Otherwise Ó0 -7 -7 (u( x, y ), v ( x, y )) = arg max u,v = -3K 3   T ( x + i, y + j; u, v ) i= 0 j = 0 † Cross Correlation -7 -7 (u, v ) = arg maxu,v = -3K 3   ( f k ( x + i, y + j )).( f k -1 ( x + i + u, y + j + v )) i= 0 j = 0 † 13 Normalized Correlation j = -7 -7   ( f ( x + i, y + j)).( f k (u, v ) = arg maxu,v = -3K 3 k -1 ( x + i + u, y + j + v )) i= 0 j = 0 -7 -7  f k -1 ( x + i + u, y + j + v ). f k -1 ( x + i + u, y + j + v ) i= 0 j = 0 † Mutual Correlation (u, v ) = arg maxu,v = -3K 3 -7 -7 1   ( f k ( x + i, y + j ) - m1).( f k-1( x + i + u, y + j + v ) - m2 ) 64s1s 2 i= 0 j = 0 † Sigma and mu are standard deviation and mean of patch-1 and patch-2 respectively 14 Issues with Correlation • Patch Size • Search Area • How many peaks Spatiotemporal Models • First order Taylor series f ( x, y, t ) = f ( x + dx, y + dy, t + dt ) f ( x, y, t ) = f ( x, y, t ) + [ X T † ∂f ∂f ∂f dx + dy + dt ∂x ∂y ∂t T fXfX X da = - X T fX f t -7 -7 (u(x, y), v(x, y)) = arg minu,v=-3K3  Â| (fk (x + i, y + j) - fk-1(x + i + u, y + j + v))| Correlation i =0 j =0 15 Bilinear and Pseudo-Perspective ( FF T )q = - f t F FT = [ f x ( xy, x, y,1), FT = [ f x ( x, y,1) c1 = x 2 f x + xyf x f y ( xy, x, y,1)] bilinear f y ( x, y,1) c1 c 2 ] Pseudo perspective c 2 = xyf x + †2 f y y † Correlation Vs Spatiotemporal Frame k-1 Frame k 8X8 16X16 16 Correlation Complexity • m*m multiplications and additions • 2*m*m additions and 2 divisions for two means • 2*m*m multiplications and additions for variances (u, v ) = arg maxu,v = -3K 3 -7 -7 1   ( f k ( x + i, y + j ) - m1).( f k-1( x + i + u, y + j + v ) - m2 ) 64s1s 2 i= 0 j = 0 † Spatiotemporal Complexity • 3* m*m subtractions for sptiotemporal derivatives • (36+6)*m*m additions for generating linear system • 6*6*6 multiplications and additions for solving 6 by 6 linear system [ X T T fXfX X da = - X T fX f t 17 Feature-based Matching Feature-based Matching • The input is formed by f1 and f2, two frames of an image sequence. • Let Q1, Q2 and Q’ be three NXN image regions. • Let “d” be the unknown displacement vector between f1 and f2 of a feature point “p”, on which Q1 is centered. 18 Algorithm • Set d=0, center Q1 on p1. • Estimate the displacement “d0” of “p”, center of “Q1”, using Lucas and Kanade method. Let d=d+d0. • Let Q’ bet the patch obtained by warping Q1 according to “d0”. Compute Sum of Square (SSD) difference between new patch Q’ and corresponding patch Q2 in frame f2. • If SSD more than threshold, set Q1=Q’ and go to step 1, otherwise exit. Lucas & Kanade (Least Squares) • Optical flow eq f xu + f y v = - f t • Consider 3 by 3 window f x1u + f y1v = - f t1 M f x 9u + f y 9 v = - f t 9 È f x1 Í ÍM Í Í Í Í f x9 Î f y1 ˘ È - f t1 ˘ Í ˙ M˙ ˙ Èu ˘ Í M ˙ ˙Í ˙ = Í ˙ ˙ Îv ˚ Í ˙ ˙ Í ˙ Í- f t 9 ˙ f y9 ˙ Î ˚ ˚ Au = f t 19 Shi-Tomasi-Kanade(STK) Tracker I (d ( x ), t + t ) = I ( x , t ) d ( x ) = Ax + d 2 e =  [I ( Ax + d , t + t ) - I ( x , t )] W After the first order Taylor expansion at A=I and d=0, we can get a linear 6_6 system: Tz = f z = [A11 [ f = I t  xI x W A21 A22 d1 d2 ] yI x xI y yI y Ix A12 Iy Shi-Tomasi-Kanade(STK) Tracker • Advantages: – Easy to implement . – Works very well with a small transformation. • Drawbacks: – Fail to track in presence of a large rotation. • Reason: ß The rotation component implied in affine model is nonlinear. 20 Useful Links http:// twtelecom.dl.sourceforge.net/sourceforge/opencvlibrary/OpenCVReferenceManual.pdf http://vision.stanford.edu/~birch/klt/. 21 ...
View Full Document

This note was uploaded on 06/12/2011 for the course CAP 6411 taught by Professor Shah during the Spring '09 term at University of Central Florida.

Ask a homework question - tutors are online