03 features

03 features - Feature extraction January 2010 READING...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Feature extraction January 2010 READING Section 4.1.1 Today Harris Corner Detector Basics of Linear filtering (to continue on Thursday) Monday, January 11, 2010 Feature extraction • why feature extraction? • what features? Monday, January 11, 2010 Today’s illusion: Rotating Rays, A. Kitaoka http://www.ritsumei.ac.jp/~akitaoka/index-e.html Monday, January 11, 2010 Consider a “stereo pair” (ignore the white lines and circles for now) 4 Monday, January 11, 2010 another example (ignore the lines) 5 Monday, January 11, 2010 Correspondence problem Left image Right image 6 Monday, January 11, 2010 Correspondence problem Left image Right image 6 Monday, January 11, 2010 Correspondence problem Left image Right image 6 Monday, January 11, 2010 Correspondence problem Left image Right image 6 Monday, January 11, 2010 Correspondence problem Left image Right image • What is a point? • How do we compare points in different images? (Similarity measure) 6 Monday, January 11, 2010 Correspondence problem 4.1. Points 215 Figure 4.3: Image pairs with extracted patches below. Notice how some patches can be localized or matched with higher accuracy than others. region around detected keypoint locations in converted into a more compact and stable (invariant) descriptor that can be matched against other descriptors. The third feature matching stage, §4.1.3, 7 efficiently searches for likely matching candidates in other images. The fourth feature tracking stage, §4.1.4, is an alternative to the third stage that only searches a small neighborhood around Monday, January 11, 2010 Aperture Problem 216 Computer Vision: Algorithms and Applications (October 18, 2009 draft) (a) (b) (c) Figure 4.4: Aperture problems for different image patches: (a) stable (“corner-like”) flow; (b) classic aperture problem (barber-pole illusion); (c) textureless region. The two images I0 (yellow) and I1 (red) are overlaid. The red vector u indicates the displacement between the patch centers, and the w(xi ) weighting function (patch window) is shown as a dark circle. 8 Ack: Szelski Chapter 4 Monday, January 11, 2010 The correspondence problem 9 Monday, January 11, 2010 The correspondence problem • A classically difficult problem in computer vision – Is every point visible in both images? – Do we match points or regions or …? – Are corresponding (L-R) image regions similar? 9 Monday, January 11, 2010 The correspondence problem • A classically difficult problem in computer vision – Is every point visible in both images? – Do we match points or regions or …? – Are corresponding (L-R) image regions similar? • The so called “aperture problem” 9 Monday, January 11, 2010 • Helpful to come prepared – read up on basic linear algebra, eigenvalues of a 2x2 matric, diagonalization etc. – will introduce you to some vision buzz-words: Gaussian kernels, linear filtering, convolution vs correlation – try to read Ch. 3 and Ch. 4 to the extent possible (even if you ignore the math details in those chapters) 10 Monday, January 11, 2010 orientation suffer from localize, as shownthe aperture problem (Horn and Schunck a. , Lucas and Kanade 1981, schematically in Figure 4.41981 Anandan 1989), i.e., it is only possible to align the patches along the direction normal to the edge Comparing image patches ionsdirectionbe formalized by looking in atthe simplest possible matching cr can (Figure 4.4b). Patches with gradients at least two (significantly) different orientations are the easiest to localize, as shown schematically in Figure 4.4a. image patches, i.e., their (weighted) summed square difference, These intuitions can be formalized by looking at the simplest possible matching criterion for comparing two image patches, i.e., their (weighted) summed square difference, 2 (4.1) EWSSD (u) = w(xi )[I1 (xi + u) − I0 (2xi )] , EWSSDi(u) = w(xi )[I1 (xi + u) − I0 (xi )] , (4.1) i arewhere I0 and imagestwo imagescompared, uu= ((u,) v )the displacement vector, w(x) vec the two I1 are the being being compared, = u, v is is the displacement is a spatially varying weighting (or window) function, and the summation i is over all the pixels in ying weighting (or window) function, later use to estimate motion betweenover all th and the summation i is complete the patch. (Note that this is the same formulation we images 8.1, and that this section shares some material with that later section.) that this§is the same formulation we later use to estimate motion between When performing feature detection, we do not know which other image location(s) the feature that thisup being matched against. Therefore, we can with that later section.) is with will end section shares some material only compute how stable this metric Using Taylor approximation: respect to small variations in we do not know which other image location(s) ming feature detection,position ∆u by comparing an image patch against itself, which is t known as an auto-correlation function or surface ) + f (x).a f (x + a) ≈ f (x g matched against. Therefore, we can only compute how stable this met EAC (∆u) = w(xi )[I0 (xi + ∆u) − I0 (xi )]2 (4.2) I0 (xi +∆u ) ≈comparing an0image patch against itself u by I0 (xi ) + I (xi ). u variations in position i (Figure 4.5).1 Note how the auto-correlation surface for the textured flower bed (Figure 4.5b, red o-correlation function or surface I (xi ) = ∂ I/ of the ∂ weighted patches; Strictly speaking, the image gradient:auto-correlation is the(product ∂ x, twoI/∂ y )(xi ) I’m using the term here in a more qualitative sense. The weighted sum of squared differences is often called an SSD surface2 8.1. § 11 EAC (∆u) = w(xi )[I0 (xi + ∆u) − I0 (xi )] 1 i Monday, January 11, 2010 ∂x ∂y (Lucas and Kanade 1981, Shi and Tomasi 1994), we can approximate the auto-co as radient at xi . This gradient can be computed using a variety of techniques The classic “Harris” detector (Harris and Stephens 1988) uses 2a [-2 -1 0 1 EAC (∆u) = ) w((xi )[0I0 (ixi + u)u) I0 (x0)]xi )]2 EAC (∆u w xi )[I (x + ∆ ∆ − − I i ( ern variants (Schmid et al. 2000,iiTriggs 2004) convolve the image with h ≈ w((x σ 0I xix). + I0 Note:∆u∆u 0− I)](do xi )[)[ ( ( 1 + [ (I i(x )Bill Triggs 2x x ) · · − I ( xi erivatives of a Gaussian (typically with i I = ) i ) ≈ w 0 0 i 0 i i i ephens (1988) used such a bad filter kernel, as reported in (Schmid et al. 2 = w(xi )[ I0 (xi ) · ∆u]2 2 ublication is hard to find. ] = i w(xi )[ I0 (xi ) · ∆u] iT = ∆u T A orrelation matrix A can be written as ∆u, = ∆u A∆u, where 2 Ix Ix Iy where A = w ∗ , ∂ I0 ∂ I0 ere 2( I0 y = , Ix I(xi ) Iy ∂ I ∂)(xi ) ∂ x 0∂ y I0 I 0 ( xi ) = ( , )(xi ) ∂x ∂y isreplaced the weighted isummations with discrete convolutions with of tech the image symmetric. matrix cancan be computed using a variety the w - A gradient at x This gradient be diagonalized. e et image ). The classic . This detector (Harris and Stephens 1988 uses a o theal. 2000gradient at xi“Harris”gradient can be computed using a) variety[-2 λ1 0 s matrix can be interpreted as tensor (multiband) image, where the outer pro but2000).modern variants A = detector (Harris and Stephens 1988)image more The classic “Harris” et al. λ (Schmid 0 2000 Triggs 2004) al. are convolved with a weighting function, w to provide convolve theestima 2 I a per-pixel uses Note: Bill tand vertical derivatives of(Schmidfunction. In ,more σ = 1).the[computationi more modern variants a Gaussian (typically with detail, ) convolve theTr et al. 2000 Triggs 2004 ic) shape of the auto-correlation eigenvalues of the matrix A. Harriswhere λ1 and λ2 are the a bad filter kernel, as reported in (Schmid and Stephens (1988) used such d2vertical derivatives of pixel can be (typically with σ steps: [ Note: B a Gaussian × 2 matrix A at each hard to find. ]performed in two = 1). the original publication is rris and Stephens (1988) used such a bad filter kernel, as reported in (Sc Monday, January 11, 2010 Local Structure Matrix • the matrix A is often referred to as the local structure matrix. • Eigenvalues of A are real and positive, and provide information about the local image structure. • Examples: – flat image: both eigenvalues will be zero. – ideal ramp edge: one eigenvalue will be non-zero and the other zero (independent of the edge orientation). – a corner will have a strong edge in the direction corresponding to the larger eigenvalue, and another edge normal to the first, and both eigenvalues will be non-zero. – for a “good” corner, both eigenvalues must be significant. – hence, conditioning on the smaller of the two eigenvalues is needed to determine a strong corner. 13 Monday, January 11, 2010 Corner response function • Computing the eigenvalues is expensive λ 1 ,2 trace(A) = ± 2 trace(A) 2 ( ) − det(A) 2 λ1 − λ2 = 2. 0.25.(trace(A))2 − det(A) • At a corner, this expression should always be positive, so one can consider the following as a corner response function: det(A) − α(trace(A)) 2 • \alpha determines the sensitivity of the detector. Larger 14 value ==> less sensitive. Typical: 0.04 - 0.06. Monday, January 11, 2010 Implementation (page 220, Oct 18 draft) 1. compute the horizontal (I_x) and vertical (I_y) derivatives of the image, by convolving the original image with derivatives of Gaussians. 2. compute the three images corresponding to the outer product of these gradients (the matrix A in the previous slides). 3. blur these three images with a larger Gaussian kernel. 4. compute a “corner response function” as discussed in the previous slide. 5. find the local maxima above a certain threshold and report these as detected feature point locations. 15 Monday, January 11, 2010 (a) Strongest 250 (b) Strongest 500 (c) ANMS 250, r = 24 (d) ANMS 500, r = 16 Monday, January 11, 2010 Project 1. Implement a Harris corner detector • There are many resources available online, include source code. For example, a matlab code for Harris detector is available at http://www.csse.uwa.edu.au/~pk/Research/ MatlabFns/. • The purpose of this project is two fold: – get your hands on programming with images while learning some basics of image processing/analysis – do something interesting, leading towards the next project • I am ok if you use online resources, but you should make sure that you acknowledge that fact. In addition, you should only use it as a starting point, to learn and implement your own version, not just copy & paste. • All project group members are expected to contribute towards the implementation equally. The grade is assigned 17 to the individual group as a whole. Monday, January 11, 2010 Project 1 (contd.) DUE JAN 22, 5PM. • you organize your report as follows: – a pdf file that includes cover page( your group number, names of the students, date of submission) and at least two pictures on which you show the results demonstrating the corner detection performance. ♦ include highlights/special features of your implementation, including an overview of the specific details that differ from the generic class discussion. ♦ discuss the effects of various parameter choices in your implementation (e.g., threshold selection, smoothing filter choices, etc. etc.) – a file that consists of the source code of your implementation. The code must be adequately documented. Any dependencies on binaries should be minimal and clearly explained. – Combine the files into a single ZIP archive with the following naming convention: Project1_Group##.zip. Insert your group number in the range 00-10. • email the ZIP archive to [email protected] Monday, January 11, 2010 18 project: final note • If you do not have significant programming experience, use MATLAB. • Programming/software specific issues are often difficult to resolve, particularly for a course such as this. Keep this in mind while choosing your software environment. – choose wisely! • In addition to the software link provided before, there are many, many, computer vision libraries that are freely available online, including OpenCV, ImageJ, etc. However, the TAs will not be providing any support for these external libraries. – again, my recommendation is to stick with the inefficient MATLAB while you learn about computer vision. 19 Monday, January 11, 2010 ...
View Full Document

Ask a homework question - tutors are online