This preview shows page 1. Sign up to view the full content.
Unformatted text preview: Ch 5  Boundary Models
•
•
•
• Overview
Edge Tracking
Active Contours
Conclusion Introduction
Overview
ppose we wish to find the boundary of an
• Suppose we wish to find the boundary of an
ject object image
in an in an image
ne • One approach is to find edge segments and
approach is to find edge segments and
link them together
k them together
Edge segments
Link these together to
make object boundary. Edge image Overview
• The process of linking edge segments to obtain
boundary models is deceptively difficult
– How do we select edge segments for the object?
– How do connect these edge segments?
– What do we do with missing or conflicting data?
– What domain knowledge can be used?
– How much user input is necessary?
– How do measure the quality of our solution? Overview
• In this section we will consider the design and
implementation of two techniques
• Edge tracking – performs a greedy search to
find a path around the object boundary
• Active contours – solves for object boundary
by minimizing an energy functional that
describes constraints on the object boundary Edge Tracking
• Edge tracking is a conceptually simple
approach for finding object boundaries
• Edge tracking algorithm:
– Start at a point (x,y) on the object boundary
– Walk clockwise around the boundary using
gradient information to choose next point
– Stop when we return to the starting point Edge Tracking User selects (x,y) starting point Edge Tracking Gradient information used to find
next (x,y) point on boundary Edge Tracking Process repeats to get sequence of
points on boundary Edge Tracking Sometimes we have to make hard
choices to pick boundary to track Edge Tracking Process continues until we reach
starting point (x,y) again Edge Tracking Or until the edge linking reaches a
dead end or the edge of the image Edge Tracking
• If we are at pixel (x,y) on the boundary how do
we select next point (x’,y’) on boundary?
• There are a number of factors to consider:
– Find point with highest gradient magnitude
– Find point with most similar gradient magnitude
– Find point with most similar gradient direction
– Find point that gives smoothest boundary Edge Tracking
• To find point with highest gradient magnitude
– Calculate gradient magnitude of image
– Search NxN neighborhood around point (x,y)
– Ignore points that are already on object boundary
– Find point (x’,y’) that has the highest gradient
magnitude value G • Very simple to implement, but might get stuck
in local gradient maxima Edge Tracking
• To find point with most similar gradient
magnitude to current boundary point
– Search NxN neighborhood around point (x,y)
– Find unused point (x’,y’) with smallest difference
in gradient magnitude M ( f x , f y ) = gradient at ( x, y )
( f x!, f y!) = gradient at ( x!, y!)
M = abs( ( f x , f y ) " ( f x!, f y!) ) Edge Tracking
• To find point with most similar gradient
direction to current boundary point
– Search NxN neighborhood around point (x,y)
– Find unused point (x’,y’) with smallest difference
in gradient direction D ( f x , f y ) = gradient at ( x, y )
( f x!, f y!) = gradient at ( x!, y!)
"1 D = cos (( f x , f y ) • ( f x!, f y!)) Edge Tracking
• To find point that gives smoothest boundary
– Search NxN neighborhood around point (x,y)
– Calculate change in direction angle relative to
current point C and previous point P on boundary
– Select point (x’,y’) with smallest angle A
135 90 45 90 45 0 P C 0 135 C 45 135 90 45 P 135 90 Edge Tracking
• We can combine edge tracking criteria by
minimizing a weighted average of scores S = w1 ! G + w2 ! M + w3 ! D + w4 ! A
• We stop edge tracking when we are close
enough to the starting point (when current
point is within radius R of starting point) Edge Tracking
• Edge tracking works well when the edge
strengths are strong in an image and the user
provides a good starting point on the edge
• Edge tracking works poorly if there are weak
or missing edges since there are no global
constraints on the shape of object boundary Active Contours
• Edge tracking approach may produce very
rough object boundaries
– No boundary smoothness constraints • Active contours solves this problem
– Define object boundary as a parametric curve
– Define smoothness constraints on the curve
– Define data matching constraints on the curve
– Deform curve to find optimal object boundary • Alternatively use an Active Contour Model
Active Contours
(or `snake’)
– A smooth curve which matches to image data
• Example below shows active contour in action
–itting itself to desired object boundary
f Curve is initialised near target
– Iteratively refined Initial position Intermediate Final position a snake to flip into a different local minimum,
but for ordinary speeds and videorate sampling,
snakes do a good job of tracking motion. Figure 8
shows eight selected frames out of a twosecond
video sequence. Edgeattracted snakes were initialized by hand on the speaker’s lips in the first
frame. After that, the snakes tracked the lip
movements automatically.
The motion tracking was done in this case
without any interframe constraints. Introducing
such constraints will doubtless make the tracking Active Contours Fig. 7. Bottom: Stereogram o f a bent piece o f paper. Below: Surface reconstruction from the outline o f the paper matched
using stereo snakes. The surface model is rendered from a
v ery different viewpoint than the original to emphasize that it
is a full 3D model, rather than a 2SD model. • Interactive active contours in operation (from
Snakes: Active contour models by Kass 1988) Fig. 8. Selected frames from a 2second video sequence show ing snakes used for motion tracking. After being initialized to the speaker’s lips in the first frame, the snakes automatically
track the lip movements with high accuracy. the upper right, the volcano can be seen bending
a nearby snake. Each of the snakes has a sharp
corner which has been specified by the user. behavior can be created. Active Contours
3.1 Line Functional 3 Image Forces In order to make snakes useful for early vision w e
need energy functionals that attract them to
salient features in images. In this section, w e present three different energy functionals which attract a snake to lines, edges, and terminations,
The total image energy can be expressed as
a weighted combination
of the three energy
functionals The simplest useful image functional
image intensity itself. If w e set
Eline = 1(X, Y) is the
(4) then depending on the sign Of Wline,the snake will
be attracted either to light lines or dark lines.
Subject to its other constraints, the snake will try
to align itself with the lightest or darkest nearby
contour. This energy functional was used with
the snakes shown in figure 1. By pushing with the • Interactive active contours in operation (from
Snakes: Active contour models by Kass 1988) Fig. 3. Two edge snakes on a pear and potato. Upperleft: The
user has pulled one o f the snakes away from the edge o f the
pear. Others: After the user lets go, the snake snaps back to the Simple ctive Contours
A curve representatio • Represent the curve set of n set of
Representthe curve with awith a points n poi vi ( xi , yi ) i 0...n 1 ( x0 , y0 )
( x3 , y3 )
( x17 , y17 ) Active Contours
• Smoothness is controlled by an internal energy
constraint on the parametric curve
• If we minimize the elastic energy the curve
will shrink like a rubber band Einternal = " vi+1 ! vi 2 i=0.. n!1
2 Einternal = ( xi+1 ! xi ) + ( yi+1 ! yi ) 2 Active Contours
• The external energy constraint describes how
well the curve matches the image data
• If we maximize the gradient magnitudes g(x,y)
along the curve we will fit the strong edges
that correspond to the object boundary Eexternal = "
i=0.. n!1 g( xi , yi ) 2 Active Contours
• We combine internal and external constraints
to obtain a total energy functional to minimize Etotal = ! Einternal ! (1 ! ! )Eexternal
• Notice that we changed the sign of external
constraint to make this a minimization problem Active Contours
• Our next task is to solve for the parametric
curve (xi, yi) that minimizes Etotal
• Consider Einternal for three consecutive points
on the curve (xi1, yi1), (xi, yi), (xi+1, yi+1)
2 Einternal = ( xi+1 ! xi ) + ( yi+1 ! yi )
2 + ( xi ! xi!1 ) + ( yi ! yi!1 ) 2 2 Active Contours
• If we take the derivative of Einternal wrt xi and
set it to zero we can solve for xi !Einternal / !xi = 0
"2 ( xi+1 " xi ) + 2 ( xi " xi"1 ) = 0
xi = ( xi"1 + xi+1 ) / 2 Active Contours
• Similarly, we take the derivative of Einternal wrt
yi and set it to zero to solve for yi !Einternal / !yi = 0
"2 ( yi+1 " yi ) + 2 ( yi " yi"1 ) = 0
yi = ( yi"1 + yi+1 ) / 2 Active Contours
• Hence we can deform the active contour to
minimize the internal energy by iteratively
averaging adjacent boundary coordinates
(xi,yi)
(xi1,yi1) (xi+1,yi+1) Active Contours
• To minimize Etotal we must consider the partial
derivatives of the Eexternal term wrt xi and yi !Eexternal / !xi = 2 g ( xi , yi ) " !g / !x ( xi , yi )
!Eexternal / !yi = 2 g ( xi , yi ) " !g / !y ( xi , yi )
• This moves the contour in the direction of the
gradient of the gradient magnitude g(x,y) Active Contours
• We can deform the active contour to minimize
the external energy by making a step of size s
in the direction of the gradient of g(xi,yi)
(xi,yi)
(xi1,yi1) (xi+1,yi+1) Active Contours
Synthetic Example
• Example – circle deforming to concave object
(1) (2) (3) (4) • Notice that the strength of the internal
Active Contours
elastic component can be controlled by a
parameter,
n1 • Example – effects of varying2i α weight in total
E internal
L
i0
energy functional • Increasing this increases stiffness of curve large medium small Dealing with missing data
Active Contours • The smoothness active contour around
• Example – fitting oneconstraint can deal with
two adjacent objects
missing data: Active Contours
• Additional constraints can be added to active
contours to introduce stiffness (minimize
second derivatives) and other external forces
(user pulling curve at selected points)
• Calculus of variations is typically used to solve
the EulerLegrange equations associated with
these active contour energy functionals Conclusions
• This section described how boundary tracking
and active contours can be used to obtain the
boundary of objects of interest in an image
• Both methods require user interaction to select
a starting point/curve
– Boundary tracking fast and easy to implement
– Active contours will result in smoother boundary ...
View
Full
Document
This note was uploaded on 12/01/2011 for the course CSCE 5013 taught by Professor Staff during the Fall '08 term at Arkansas.
 Fall '08
 STAFF

Click to edit the document details