optical flow handout

optical flow handout - i 192 Chapter 8 Motion 8.3.1 The...

Info iconThis preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 4
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: i 192 Chapter 8 Motion 8.3.1 The Image Brightness Constancy Equation It is common experience that, under most circumstances, the apparent brightness of moving objects remains constant. We have seen in Chapter 2 that the image irradiance is proportional to the scene radiance in the direction of the optical axis of the camera; if we assume that the proportionality factor is the same across the entire image plane, the constancy of the apparent brightness of the observed scene can be written as the stationarity of the image brightness E over time: dB 0': _ W“ In (8.15). the image brightness. E, should be regarded as a function of both the spatial coordinates of the image plane, it and y. and of time, that is, E = E(x. y. r). Since x and ,v are in turn functions of I, the total derivative in (8.15) should not be confused with the partial derivative BIZ/8r. 0. (8.15) Via the chain rule of differentiation, the total temporal derivative reads dE[x(t)_. v0), I) BE dx BE dy BE —..' = _ — — — — : 0, at 3x 0': + By d: + 3; ( 6) The partial spatial derivatives of the image brightness are simply the components of the spatial image gradient, VE, and the temporal derivatives, dx/dr and dyfdr, the components of the motion field, v. Using these facts, we can rewrite (8.16) as the image brightness constancy equation. The Image Brightness Constancy Equation Given the image brightness, E = E(x, y, I). and the motion field, v, (VE)Tv + E, = 0. [8.17) The subscript I denotes partial differentiation with respect to time. We shall now discuss the relevance and applicability of this equation for the estimation of the motion field. 8.3.2 The Aperture Problem How much of the motion field can be determined through (8.17)? Onl'y its component in the direction ofrhe siclai‘r'atr image gnraaiem,9 vn. We can see this analytically by isolating the measurable quantities in (8.11): E, _ {VE)Tv _ __ 2,, 8.18 [IVEII JIVEII ” ( ) 9This component is sometimes called the normal component, because the spatial image gradient is normal to the spatial direction along which image intensity remains constant. Section 8.3 The Notion of Optical Flow 193 ltness of radiance camera; ge plane, :n as the (8.15) 16 Spatial toe x and with the (a) (b) : Figure 8.7 The aperture problem: the black and grey lines show two positions of the same image line in two consecutive frames. The image velocity perceived in (a) (8.16) _ . . . through the small aperture, vn, 15 only the component parallel to the image gradient of the true image velocity, v, revealed in (b). nents of fdt, the .e image —-—_—___—______— The Aperture Problem The component of the motion field in the direction orthogonal to the spatial image gradient is not constrained by the image brightness constancy equation. m (8.17) The aperture problem can be visualized as follows. Imagine to observe a thin, black rectangle moving against a white background through a small aperture. “Small” means that the corners of the rectangle are not visible through the aperture (Figure 8.7(a)); the small aperture simulates the narrow support of a differential method. Clearly, there for the _ are many, actually infinite, motions of the rectangle compatible with what you see through the aperture (Figure 83%)); the visual information available is only sufficient to determine the velocity in the direction orthogonal to the visible side of the rectangle; the velocity in the parallel direction cannot be estimated. iponent l t, 1%” Notice that the parallel between (8.1?) and Figure 8.7 is not perfect. Equation (8.1?) relates ;o a trig the image gradient and the motion field at the same image point. thereby establishing a constraint on an infinitely small spatial support: instead, Figure 8.7 describes a state of affairs over a small but finite spatial region. This immediately suggests that a possible (8.18) strategy for solving the aperture problem is to look at the spatial and temporal variations . of the image brightness over a neighborhood of each point.10 tormal to w Incidentally. this strategy appears to be adopted by the visual system of primates 196 Chapters Motion _ We describe a differential technique that gives good results. The basic assumption 5 is that the motion field is well approximated by a constant vector field, v, within any '. small region of the image plane.11 __________.—‘—-——e—— Assumptions L The image brightness constancy equation yields a good approximation of the normal component of the motion field. 2. The motion field is well approximated by a constant vector field within any small patch of the image plane. __—________._.—.——————-— An Optical Flow Algorithm. Given Assumption 1, for each point [3.- within a small, N x N patch, Q, we can write (VEiTv + E, = n where the spatial and temporal derivatives of the image brightness are computed at PIePZ - - - PNE- I? A typical size of the “small patch" is 5 x 5. that minimizes the functional on] = Z [(VEJTv + 5,]2. PI'EQ | | | | l | i ‘ Therefore, the optical flow can be estimated within Q as the constant vector, v, 1 It | l l: The solution to this least squares problem can be found by solving the linear system i l ATAv = ATIJ. (8.22) The i—th row of the N2 x 2 matrix A is the spatial image gradient evaluated at point 11,-: VEtPi) V5032) A = ’ , (8.23) . VEtpMN) l. and h is the NZ-dimensional vector of the partial temporal derivatives of the image 2; brightness, evaluated at p], . . . pNz, after a sign change: '1 = - [Emit]. . . . . Enpwxmf. (8.24) _ “ Notice that this is in agreement with the first conclusion of section 8.2.3 (motion field of moving planes) 5 regarding the approximation of smooth motion fields. assumption within any the normal 1ail patch of p,- within a mputed at vector, \7, system (8.22) t point Pg: (823) he image (824) 'iug planes) Section 8.4 Estimating the Motion Field 197 'lhe least squares solution of the overconstrained system (8.22) can be obtained as12 v=(ATA)—‘ATIJ. (8.25) v is the optical flow (the estimate of the motion field) at the center ot'patch Q; repeating this procedure for all image points, we obtain a dense optical flow. We summarize the algorithm as follows: Algorithm CONSTANT__FLOW The input is a time-varying sequence of n images. E1. E2. . . . E”. Let Q be a square region of N x N pixels (typically. N = 5). 1. Filter each image of the sequence with a Gaussian filter of standard deviation equal to a. (typically a, = 1.5 pixels} along each spatial dimension. 2. Filter each image of the sequence along the temporal dimension with a Gaussian filter of standard deviation a, (typically a, :15 frames). If 2!: + l is the size of the temporal filter. leave out the first and last k images. 3. For each pixel of each image of the sequence: (a) compute the matrix A and the vector 11 using (8.23) and (8.24) (b) compute the optical flow using (8.25) The output is the optical flow computed in the last step. It? The purpose of spatial filtering is to attenuate noise in the estimation of the spatial image gradient: temporal filtering prevents aliasing in the time domain. For the implementation of the temporal filtering, imagine to stack the images one on top of the other. and filter sequences of pixels having the same coordinates. Note that the size of the temporal filter is linked to the maximum speed that can be “measured” by the algorithm. An Improved Optical Flow Algorithm. We can improve CONSTANT_F LOW by observing that the error made by approximating the motion field at p with its estimate at the center of a patch increases with the distance of p from the center itself. This suggests a weighted least-square algorithm, in which the points close to the center of the patch are given more weight than those at the periphery. If W is the weight matrix‘ the solution, a... is given by vw=(ATW2A)‘1ATW2b. Concluding Remarks on Optical Flow Methods. It is instructive to examine the image locations at which CONSTANT_FLOW fails. As we have seen in Chapter 4, the 2 x 2 matrix ATA=( 2e sap), Z HIE). E E? (8.26) 12 See Appendix. section A6 for alternative ways of solving overconstrained linear systems ...
View Full Document

This note was uploaded on 02/14/2008 for the course CSE 166 taught by Professor Belongie during the Fall '06 term at UCSD.

Page1 / 4

optical flow handout - i 192 Chapter 8 Motion 8.3.1 The...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online