This preview shows pages 1–4. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: i 192 Chapter 8 Motion 8.3.1 The Image Brightness Constancy Equation It is common experience that, under most circumstances, the apparent brightness of
moving objects remains constant. We have seen in Chapter 2 that the image irradiance
is proportional to the scene radiance in the direction of the optical axis of the camera;
if we assume that the proportionality factor is the same across the entire image plane,
the constancy of the apparent brightness of the observed scene can be written as the
stationarity of the image brightness E over time: dB
0': _
W“ In (8.15). the image brightness. E, should be regarded as a function of both the spatial
coordinates of the image plane, it and y. and of time, that is, E = E(x. y. r). Since x and ,v are in turn functions of I, the total derivative in (8.15) should not be confused with the
partial derivative BIZ/8r. 0. (8.15) Via the chain rule of differentiation, the total temporal derivative reads dE[x(t)_. v0), I) BE dx BE dy BE
—..' = _ — — — — : 0, at 3x 0': + By d: + 3; ( 6)
The partial spatial derivatives of the image brightness are simply the components of
the spatial image gradient, VE, and the temporal derivatives, dx/dr and dyfdr, the
components of the motion ﬁeld, v. Using these facts, we can rewrite (8.16) as the image brightness constancy equation. The Image Brightness Constancy Equation
Given the image brightness, E = E(x, y, I). and the motion ﬁeld, v,
(VE)Tv + E, = 0. [8.17) The subscript I denotes partial differentiation with respect to time. We shall now discuss the relevance and applicability of this equation for the
estimation of the motion ﬁeld. 8.3.2 The Aperture Problem How much of the motion ﬁeld can be determined through (8.17)? Onl'y its component
in the direction ofrhe siclai‘r'atr image gnraaiem,9 vn. We can see this analytically by isolating
the measurable quantities in (8.11): E, _ {VE)Tv _ __ 2,, 8.18
[IVEII JIVEII ” ( ) 9This component is sometimes called the normal component, because the spatial image gradient is normal to
the spatial direction along which image intensity remains constant. Section 8.3 The Notion of Optical Flow 193 ltness of
radiance
camera;
ge plane,
:n as the
(8.15)
16 Spatial toe x and with the (a) (b) :
Figure 8.7 The aperture problem: the black and grey lines show two positions of
the same image line in two consecutive frames. The image velocity perceived in (a)
(8.16) _ . . .
through the small aperture, vn, 15 only the component parallel to the image gradient of
the true image velocity, v, revealed in (b).
nents of
fdt, the
.e image
——_—___—______—
The Aperture Problem
The component of the motion ﬁeld in the direction orthogonal to the spatial image gradient is
not constrained by the image brightness constancy equation.
m
(8.17) The aperture problem can be visualized as follows. Imagine to observe a thin, black rectangle moving against a white background through a small aperture. “Small” means
that the corners of the rectangle are not visible through the aperture (Figure 8.7(a));
the small aperture simulates the narrow support of a differential method. Clearly, there
for the _ are many, actually inﬁnite, motions of the rectangle compatible with what you see
through the aperture (Figure 83%)); the visual information available is only sufﬁcient to determine the velocity in the direction orthogonal to the visible side of the rectangle;
the velocity in the parallel direction cannot be estimated. iponent l t, 1%” Notice that the parallel between (8.1?) and Figure 8.7 is not perfect. Equation (8.1?) relates
;o a trig the image gradient and the motion ﬁeld at the same image point. thereby establishing
a constraint on an inﬁnitely small spatial support: instead, Figure 8.7 describes a state
of affairs over a small but ﬁnite spatial region. This immediately suggests that a possible (8.18) strategy for solving the aperture problem is to look at the spatial and temporal variations
. of the image brightness over a neighborhood of each point.10 tormal to w Incidentally. this strategy appears to be adopted by the visual system of primates 196 Chapters Motion _ We describe a differential technique that gives good results. The basic assumption
5 is that the motion ﬁeld is well approximated by a constant vector ﬁeld, v, within any
'. small region of the image plane.11 __________.—‘———e—— Assumptions L The image brightness constancy equation yields a good approximation of the normal
component of the motion ﬁeld. 2. The motion ﬁeld is well approximated by a constant vector ﬁeld within any small patch of
the image plane. __—________._.—.——————— An Optical Flow Algorithm. Given Assumption 1, for each point [3. within a
small, N x N patch, Q, we can write (VEiTv + E, = n where the spatial and temporal derivatives of the image brightness are computed at
PIePZ    PNE I? A typical size of the “small patch" is 5 x 5. that minimizes the functional on] = Z [(VEJTv + 5,]2. PI'EQ 



l

i ‘ Therefore, the optical ﬂow can be estimated within Q as the constant vector, v,
1
It
 l
l: The solution to this least squares problem can be found by solving the linear system i l ATAv = ATIJ. (8.22) The i—th row of the N2 x 2 matrix A is the spatial image gradient evaluated at point 11,: VEtPi)
V5032)
A = ’ , (8.23)
. VEtpMN)
l. and h is the NZdimensional vector of the partial temporal derivatives of the image
2; brightness, evaluated at p], . . . pNz, after a sign change:
'1 =  [Emit]. . . . . Enpwxmf. (8.24) _ “ Notice that this is in agreement with the ﬁrst conclusion of section 8.2.3 (motion ﬁeld of moving planes)
5 regarding the approximation of smooth motion ﬁelds. assumption
within any the normal 1ail patch of p, within a mputed at vector, \7, system
(8.22) t point Pg: (823) he image (824) 'iug planes) Section 8.4 Estimating the Motion Field 197 'lhe least squares solution of the overconstrained system (8.22) can be obtained as12
v=(ATA)—‘ATIJ. (8.25) v is the optical ﬂow (the estimate of the motion ﬁeld) at the center ot'patch Q; repeating this procedure for all image points, we obtain a dense optical ﬂow. We summarize the
algorithm as follows: Algorithm CONSTANT__FLOW The input is a timevarying sequence of n images. E1. E2. . . . E”. Let Q be a square region of
N x N pixels (typically. N = 5). 1. Filter each image of the sequence with a Gaussian ﬁlter of standard deviation equal to a.
(typically a, = 1.5 pixels} along each spatial dimension. 2. Filter each image of the sequence along the temporal dimension with a Gaussian filter of standard deviation a, (typically a, :15 frames). If 2!: + l is the size of the temporal ﬁlter.
leave out the ﬁrst and last k images. 3. For each pixel of each image of the sequence: (a) compute the matrix A and the vector 11 using (8.23) and (8.24)
(b) compute the optical ﬂow using (8.25) The output is the optical ﬂow computed in the last step. It? The purpose of spatial ﬁltering is to attenuate noise in the estimation of the spatial image
gradient: temporal ﬁltering prevents aliasing in the time domain. For the implementation
of the temporal ﬁltering, imagine to stack the images one on top of the other. and ﬁlter
sequences of pixels having the same coordinates. Note that the size of the temporal ﬁlter
is linked to the maximum speed that can be “measured” by the algorithm. An Improved Optical Flow Algorithm. We can improve CONSTANT_F LOW
by observing that the error made by approximating the motion ﬁeld at p with its estimate
at the center of a patch increases with the distance of p from the center itself. This
suggests a weighted leastsquare algorithm, in which the points close to the center of the patch are given more weight than those at the periphery. If W is the weight matrix‘
the solution, a... is given by vw=(ATW2A)‘1ATW2b. Concluding Remarks on Optical Flow Methods. It is instructive to examine the image locations at which CONSTANT_FLOW fails. As we have seen in Chapter 4, the
2 x 2 matrix ATA=( 2e sap), Z HIE). E E? (8.26) 12 See Appendix. section A6 for alternative ways of solving overconstrained linear systems ...
View
Full
Document
This note was uploaded on 02/14/2008 for the course CSE 166 taught by Professor Belongie during the Fall '06 term at UCSD.
 Fall '06
 Belongie
 Image processing

Click to edit the document details