ch05 - Ch 5 - Boundary Models •  •  •  • ...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Ch 5 - Boundary Models •  •  •  •  Overview Edge Tracking Active Contours Conclusion Introduction Overview ppose we wish to find the boundary of an •  Suppose we wish to find the boundary of an ject object image in an in an image ne •  One approach is to find edge segments and approach is to find edge segments and link them together k them together Edge segments Link these together to make object boundary. Edge image Overview •  The process of linking edge segments to obtain boundary models is deceptively difficult –  How do we select edge segments for the object? –  How do connect these edge segments? –  What do we do with missing or conflicting data? –  What domain knowledge can be used? –  How much user input is necessary? –  How do measure the quality of our solution? Overview •  In this section we will consider the design and implementation of two techniques •  Edge tracking – performs a greedy search to find a path around the object boundary •  Active contours – solves for object boundary by minimizing an energy functional that describes constraints on the object boundary Edge Tracking •  Edge tracking is a conceptually simple approach for finding object boundaries •  Edge tracking algorithm: –  Start at a point (x,y) on the object boundary –  Walk clockwise around the boundary using gradient information to choose next point –  Stop when we return to the starting point Edge Tracking User selects (x,y) starting point Edge Tracking Gradient information used to find next (x,y) point on boundary Edge Tracking Process repeats to get sequence of points on boundary Edge Tracking Sometimes we have to make hard choices to pick boundary to track Edge Tracking Process continues until we reach starting point (x,y) again Edge Tracking Or until the edge linking reaches a dead end or the edge of the image Edge Tracking •  If we are at pixel (x,y) on the boundary how do we select next point (x’,y’) on boundary? •  There are a number of factors to consider: –  Find point with highest gradient magnitude –  Find point with most similar gradient magnitude –  Find point with most similar gradient direction –  Find point that gives smoothest boundary Edge Tracking •  To find point with highest gradient magnitude –  Calculate gradient magnitude of image –  Search NxN neighborhood around point (x,y) –  Ignore points that are already on object boundary –  Find point (x’,y’) that has the highest gradient magnitude value G •  Very simple to implement, but might get stuck in local gradient maxima Edge Tracking •  To find point with most similar gradient magnitude to current boundary point –  Search NxN neighborhood around point (x,y) –  Find unused point (x’,y’) with smallest difference in gradient magnitude M ( f x , f y ) = gradient at ( x, y ) ( f x!, f y!) = gradient at ( x!, y!) M = abs( ( f x , f y ) " ( f x!, f y!) ) Edge Tracking •  To find point with most similar gradient direction to current boundary point –  Search NxN neighborhood around point (x,y) –  Find unused point (x’,y’) with smallest difference in gradient direction D ( f x , f y ) = gradient at ( x, y ) ( f x!, f y!) = gradient at ( x!, y!) "1 D = cos (( f x , f y ) • ( f x!, f y!)) Edge Tracking •  To find point that gives smoothest boundary –  Search NxN neighborhood around point (x,y) –  Calculate change in direction angle relative to current point C and previous point P on boundary –  Select point (x’,y’) with smallest angle A 135 90 45 90 45 0 P C 0 135 C 45 135 90 45 P 135 90 Edge Tracking •  We can combine edge tracking criteria by minimizing a weighted average of scores S = w1 ! G + w2 ! M + w3 ! D + w4 ! A •  We stop edge tracking when we are close enough to the starting point (when current point is within radius R of starting point) Edge Tracking •  Edge tracking works well when the edge strengths are strong in an image and the user provides a good starting point on the edge •  Edge tracking works poorly if there are weak or missing edges since there are no global constraints on the shape of object boundary Active Contours •  Edge tracking approach may produce very rough object boundaries –  No boundary smoothness constraints •  Active contours solves this problem –  Define object boundary as a parametric curve –  Define smoothness constraints on the curve –  Define data matching constraints on the curve –  Deform curve to find optimal object boundary • Alternatively use an Active Contour Model Active Contours (or `snake’) – A smooth curve which matches to image data •  Example below shows active contour in action –itting itself to desired object boundary f Curve is initialised near target – Iteratively refined Initial position Intermediate Final position a snake to flip into a different local minimum, but for ordinary speeds and video-rate sampling, snakes do a good job of tracking motion. Figure 8 shows eight selected frames out of a two-second video sequence. Edge-attracted snakes were initialized by hand on the speaker’s lips in the first frame. After that, the snakes tracked the lip movements automatically. The motion tracking was done in this case without any interframe constraints. Introducing such constraints will doubtless make the tracking Active Contours Fig. 7. Bottom: Stereogram o f a bent piece o f paper. Below: Surface reconstruction from the outline o f the paper matched using stereo snakes. The surface model is rendered from a v ery different viewpoint than the original to emphasize that it is a full 3D model, rather than a 2SD model. •  Interactive active contours in operation (from Snakes: Active contour models by Kass 1988) Fig. 8. Selected frames from a 2-second video sequence show- ing snakes used for motion tracking. After being initialized to the speaker’s lips in the first frame, the snakes automatically track the lip movements with high accuracy. the upper right, the volcano can be seen bending a nearby snake. Each of the snakes has a sharp corner which has been specified by the user. behavior can be created. Active Contours 3.1 Line Functional 3 Image Forces In order to make snakes useful for early vision w e need energy functionals that attract them to salient features in images. In this section, w e present three different energy functionals which attract a snake to lines, edges, and terminations, The total image energy can be expressed as a weighted combination of the three energy functionals The simplest useful image functional image intensity itself. If w e set Eline = 1(X, Y) is the (4) then depending on the sign Of Wline,the snake will be attracted either to light lines or dark lines. Subject to its other constraints, the snake will try to align itself with the lightest or darkest nearby contour. This energy functional was used with the snakes shown in figure 1. By pushing with the •  Interactive active contours in operation (from Snakes: Active contour models by Kass 1988) Fig. 3. Two edge snakes on a pear and potato. Upper-left: The user has pulled one o f the snakes away from the edge o f the pear. Others: After the user lets go, the snake snaps back to the Simple ctive Contours A curve representatio •  Represent the curve set of n set of Representthe curve with awith a points n poi vi ( xi , yi ) i 0...n 1 ( x0 , y0 ) ( x3 , y3 ) ( x17 , y17 ) Active Contours •  Smoothness is controlled by an internal energy constraint on the parametric curve •  If we minimize the elastic energy the curve will shrink like a rubber band Einternal = " vi+1 ! vi 2 i=0.. n!1 2 Einternal = ( xi+1 ! xi ) + ( yi+1 ! yi ) 2 Active Contours •  The external energy constraint describes how well the curve matches the image data •  If we maximize the gradient magnitudes g(x,y) along the curve we will fit the strong edges that correspond to the object boundary Eexternal = " i=0.. n!1 g( xi , yi ) 2 Active Contours •  We combine internal and external constraints to obtain a total energy functional to minimize Etotal = ! Einternal ! (1 ! ! )Eexternal •  Notice that we changed the sign of external constraint to make this a minimization problem Active Contours •  Our next task is to solve for the parametric curve (xi, yi) that minimizes Etotal •  Consider Einternal for three consecutive points on the curve (xi-1, yi-1), (xi, yi), (xi+1, yi+1) 2 Einternal = ( xi+1 ! xi ) + ( yi+1 ! yi ) 2 + ( xi ! xi!1 ) + ( yi ! yi!1 ) 2 2 Active Contours •  If we take the derivative of Einternal wrt xi and set it to zero we can solve for xi !Einternal / !xi = 0 "2 ( xi+1 " xi ) + 2 ( xi " xi"1 ) = 0 xi = ( xi"1 + xi+1 ) / 2 Active Contours •  Similarly, we take the derivative of Einternal wrt yi and set it to zero to solve for yi !Einternal / !yi = 0 "2 ( yi+1 " yi ) + 2 ( yi " yi"1 ) = 0 yi = ( yi"1 + yi+1 ) / 2 Active Contours •  Hence we can deform the active contour to minimize the internal energy by iteratively averaging adjacent boundary coordinates (xi,yi) (xi-1,yi-1) (xi+1,yi+1) Active Contours •  To minimize Etotal we must consider the partial derivatives of the Eexternal term wrt xi and yi !Eexternal / !xi = 2 g ( xi , yi ) " !g / !x ( xi , yi ) !Eexternal / !yi = 2 g ( xi , yi ) " !g / !y ( xi , yi ) •  This moves the contour in the direction of the gradient of the gradient magnitude g(x,y) Active Contours •  We can deform the active contour to minimize the external energy by making a step of size s in the direction of the gradient of g(xi,yi) (xi,yi) (xi-1,yi-1) (xi+1,yi+1) Active Contours Synthetic Example •  Example – circle deforming to concave object (1) (2) (3) (4) • Notice that the strength of the internal Active Contours elastic component can be controlled by a parameter, n1 •  Example – effects of varying2i α weight in total E internal L i0 energy functional • Increasing this increases stiffness of curve large medium small Dealing with missing data Active Contours • The smoothness active contour around •   Example – fitting oneconstraint can deal with two adjacent objects missing data: Active Contours •  Additional constraints can be added to active contours to introduce stiffness (minimize second derivatives) and other external forces (user pulling curve at selected points) •  Calculus of variations is typically used to solve the Euler-Legrange equations associated with these active contour energy functionals Conclusions •  This section described how boundary tracking and active contours can be used to obtain the boundary of objects of interest in an image •  Both methods require user interaction to select a starting point/curve –  Boundary tracking fast and easy to implement –  Active contours will result in smoother boundary ...
View Full Document

This note was uploaded on 12/01/2011 for the course CSCE 5013 taught by Professor Staff during the Fall '08 term at Arkansas.

Ask a homework question - tutors are online