Unformatted text preview: Feature extraction
January 2010 READING
Section 4.1.1
Today
Harris Corner Detector
Basics of Linear ﬁltering (to continue on Thursday) Monday, January 11, 2010 Feature extraction
• why feature extraction?
• what features? Monday, January 11, 2010 Today’s illusion: Rotating Rays, A. Kitaoka
http://www.ritsumei.ac.jp/~akitaoka/indexe.html Monday, January 11, 2010 Consider a “stereo pair” (ignore the white lines and circles for now) 4 Monday, January 11, 2010 another example (ignore the lines) 5 Monday, January 11, 2010 Correspondence problem Left image Right image 6 Monday, January 11, 2010 Correspondence problem Left image Right image 6 Monday, January 11, 2010 Correspondence problem Left image Right image 6 Monday, January 11, 2010 Correspondence problem Left image Right image 6 Monday, January 11, 2010 Correspondence problem Left image Right image • What is a point?
• How do we compare points in different images? (Similarity measure)
6 Monday, January 11, 2010 Correspondence problem
4.1. Points 215 Figure 4.3: Image pairs with extracted patches below. Notice how some patches can be localized
or matched with higher accuracy than others.
region around detected keypoint locations in converted into a more compact and stable (invariant)
descriptor that can be matched against other descriptors. The third feature matching stage, §4.1.3,
7
efﬁciently searches for likely matching candidates in other images. The fourth feature tracking
stage, §4.1.4, is an alternative to the third stage that only searches a small neighborhood around Monday, January 11, 2010 Aperture Problem
216 Computer Vision: Algorithms and Applications (October 18, 2009 draft) (a) (b) (c) Figure 4.4: Aperture problems for different image patches: (a) stable (“cornerlike”) ﬂow; (b)
classic aperture problem (barberpole illusion); (c) textureless region. The two images I0 (yellow)
and I1 (red) are overlaid. The red vector u indicates the displacement between the patch centers,
and the w(xi ) weighting function (patch window) is shown as a dark circle.
8
Ack: Szelski Chapter 4
Monday, January 11, 2010 The correspondence problem 9 Monday, January 11, 2010 The correspondence problem
• A classically difficult problem in computer vision
– Is every point visible in both images?
– Do we match points or regions or …?
– Are corresponding (LR) image regions similar? 9 Monday, January 11, 2010 The correspondence problem
• A classically difficult problem in computer vision
– Is every point visible in both images?
– Do we match points or regions or …?
– Are corresponding (LR) image regions similar? • The so called “aperture problem” 9 Monday, January 11, 2010 • Helpful to come prepared
– read up on basic linear algebra, eigenvalues of a 2x2 matric,
diagonalization etc.
– will introduce you to some vision buzzwords: Gaussian kernels,
linear filtering, convolution vs correlation
– try to read Ch. 3 and Ch. 4 to the extent possible (even if you ignore
the math details in those chapters) 10 Monday, January 11, 2010 orientation suffer from
localize, as shownthe aperture problem (Horn and Schunck a. , Lucas and Kanade 1981,
schematically in Figure 4.41981
Anandan 1989), i.e., it is only possible to align the patches along the direction normal to the edge
Comparing image patches
ionsdirectionbe formalized by looking in atthe simplest possible matching cr
can (Figure 4.4b). Patches with gradients at least two (signiﬁcantly) different orientations
are the easiest to localize, as shown schematically in Figure 4.4a.
image patches, i.e., their (weighted) summed square difference,
These intuitions can be formalized by looking at the simplest possible matching criterion for
comparing two image patches, i.e., their (weighted) summed square difference,
2
(4.1)
EWSSD (u) =
w(xi )[I1 (xi + u) − I0 (2xi )] ,
EWSSDi(u) =
w(xi )[I1 (xi + u) − I0 (xi )] ,
(4.1)
i arewhere I0 and imagestwo imagescompared, uu= ((u,) v )the displacement vector, w(x) vec
the two I1 are the being being compared, = u, v is is the displacement
is a spatially varying weighting (or window) function, and the summation i is over all the pixels in
ying weighting (or window) function, later use to estimate motion betweenover all th
and the summation i is complete
the patch. (Note that this is the same formulation we
images 8.1, and that this section shares some material with that later section.)
that this§is the same formulation we later use to estimate motion between
When performing feature detection, we do not know which other image location(s) the feature
that thisup being matched against. Therefore, we can with that later section.) is with
will end section shares some material only compute how stable this metric
Using Taylor approximation:
respect to small variations in we do not know which other image location(s)
ming feature detection,position ∆u by comparing an image patch against itself, which is t
known as an autocorrelation function or surface ) + f (x).a
f (x + a) ≈ f (x
g matched against. Therefore, we can only compute how stable this met
EAC (∆u) =
w(xi )[I0 (xi + ∆u) − I0 (xi )]2
(4.2)
I0 (xi +∆u ) ≈comparing an0image patch against itself
u by I0 (xi ) + I (xi ). u
variations in position
i
(Figure 4.5).1 Note how the autocorrelation surface for the textured ﬂower bed (Figure 4.5b, red
ocorrelation function or surface
I (xi ) = ∂ I/ of the ∂ weighted patches;
Strictly speaking, the
image gradient:autocorrelation is the(product ∂ x, twoI/∂ y )(xi ) I’m using the term here in a
more qualitative sense. The weighted sum of squared differences is often called an SSD surface2 8.1.
§ 11
EAC (∆u) =
w(xi )[I0 (xi + ∆u) − I0 (xi )]
1 i Monday, January 11, 2010 ∂x ∂y
(Lucas and Kanade 1981, Shi and Tomasi 1994), we can approximate the autoco
as
radient at xi . This gradient can be computed using a variety of techniques The classic “Harris” detector (Harris and Stephens 1988) uses 2a [2 1 0 1
EAC (∆u) =
)
w((xi )[0I0 (ixi + u)u) I0 (x0)]xi )]2
EAC (∆u
w xi )[I (x + ∆ ∆ − − I i (
ern variants (Schmid et al. 2000,iiTriggs 2004) convolve the image with h
≈
w((x σ 0I xix). + I0 Note:∆u∆u 0− I)](do
xi )[)[ ( ( 1 + [ (I i(x )Bill Triggs 2x
x ) · · − I ( xi
erivatives of a Gaussian (typically with i I = ) i )
≈
w
0
0
i
0
i
i
i
ephens (1988) used such a bad ﬁlter kernel, as reported in (Schmid et al. 2
=
w(xi )[ I0 (xi ) · ∆u]2 2
ublication is hard to ﬁnd. ] = i w(xi )[ I0 (xi ) · ∆u]
iT
= ∆u T
A
orrelation matrix A can be written as ∆u,
= ∆u A∆u, where 2
Ix Ix Iy where A = w ∗ ,
∂ I0 ∂ I0
ere
2(
I0 y
=
,
Ix I(xi ) Iy ∂ I ∂)(xi )
∂ x 0∂ y I0
I 0 ( xi ) = (
,
)(xi )
∂x ∂y
isreplaced the weighted isummations with discrete convolutions with of tech
the image symmetric. matrix cancan be computed using a variety the w
 A gradient at x This gradient be diagonalized.
e
et image ). The classic . This detector (Harris and Stephens 1988 uses a o
theal. 2000gradient at xi“Harris”gradient can be computed using a) variety[2
λ1 0
s matrix can be interpreted as tensor (multiband) image, where the outer pro
but2000).modern variants A = detector (Harris and Stephens 1988)image
more The classic “Harris” et al. λ
(Schmid 0
2000 Triggs 2004)
al. are convolved with a weighting function, w to provide convolve theestima
2
I
a perpixel uses
Note: Bill
tand vertical derivatives of(Schmidfunction. In ,more σ = 1).the[computationi
more modern variants a Gaussian (typically with detail, ) convolve theTr
et al. 2000 Triggs 2004
ic) shape of the autocorrelation eigenvalues of the matrix A.
Harriswhere λ1 and λ2 are the a bad ﬁlter kernel, as reported in (Schmid
and Stephens (1988) used such
d2vertical derivatives of pixel can be (typically with σ steps: [ Note: B
a Gaussian
× 2 matrix A at each hard to ﬁnd. ]performed in two = 1).
the original publication is
rris and Stephens (1988) used such a bad ﬁlter kernel, as reported in (Sc
Monday, January 11, 2010 Local Structure Matrix
• the matrix A is often referred to as the local structure matrix.
• Eigenvalues of A are real and positive, and provide
information about the local image structure.
• Examples:
– flat image: both eigenvalues will be zero.
– ideal ramp edge: one eigenvalue will be nonzero and the other zero
(independent of the edge orientation).
– a corner will have a strong edge in the direction corresponding to the
larger eigenvalue, and another edge normal to the first, and both
eigenvalues will be nonzero.
– for a “good” corner, both eigenvalues must be significant.
– hence, conditioning on the smaller of the two eigenvalues is needed to
determine a strong corner.
13 Monday, January 11, 2010 Corner response function
• Computing the eigenvalues is expensive λ 1 ,2 trace(A)
=
±
2 trace(A) 2
(
) − det(A)
2 λ1 − λ2 = 2. 0.25.(trace(A))2 − det(A)
• At a corner, this expression should always be positive,
so one can consider the following as a corner response
function: det(A) − α(trace(A)) 2 • \alpha determines the sensitivity of the detector. Larger
14
value ==> less sensitive. Typical: 0.04  0.06.
Monday, January 11, 2010 Implementation (page 220, Oct 18 draft)
1. compute the horizontal (I_x) and vertical (I_y) derivatives
of the image, by convolving the original image with
derivatives of Gaussians.
2. compute the three images corresponding to the outer product
of these gradients (the matrix A in the previous slides).
3. blur these three images with a larger Gaussian kernel.
4. compute a “corner response function” as discussed in the
previous slide.
5. find the local maxima above a certain threshold and report
these as detected feature point locations. 15 Monday, January 11, 2010 (a) Strongest 250 (b) Strongest 500 (c) ANMS 250, r = 24 (d) ANMS 500, r = 16 Monday, January 11, 2010 Project 1. Implement a Harris corner detector
• There are many resources available online, include source
code. For example, a matlab code for Harris detector is
available at http://www.csse.uwa.edu.au/~pk/Research/
MatlabFns/.
• The purpose of this project is two fold:
– get your hands on programming with images while learning some
basics of image processing/analysis
– do something interesting, leading towards the next project • I am ok if you use online resources, but you should make
sure that you acknowledge that fact. In addition, you should
only use it as a starting point, to learn and implement your
own version, not just copy & paste.
• All project group members are expected to contribute
towards the implementation equally. The grade is assigned
17
to the individual group as a whole.
Monday, January 11, 2010 Project 1 (contd.) DUE JAN 22, 5PM.
• you organize your report as follows:
– a pdf file that includes cover page( your group number, names of the
students, date of submission) and at least two pictures on which you
show the results demonstrating the corner detection performance.
♦ include highlights/special features of your implementation,
including an overview of the specific details that differ from the
generic class discussion.
♦ discuss the effects of various parameter choices in your
implementation (e.g., threshold selection, smoothing filter choices,
etc. etc.)
– a file that consists of the source code of your implementation. The
code must be adequately documented. Any dependencies on binaries
should be minimal and clearly explained.
– Combine the files into a single ZIP archive with the following naming
convention: Project1_Group##.zip. Insert your group number in the
range 0010. • email the ZIP archive to [email protected]
Monday, January 11, 2010 18 project: final note
• If you do not have significant programming experience, use
MATLAB.
• Programming/software specific issues are often difficult to
resolve, particularly for a course such as this. Keep this in
mind while choosing your software environment.
– choose wisely! • In addition to the software link provided before, there are
many, many, computer vision libraries that are freely
available online, including OpenCV, ImageJ, etc. However,
the TAs will not be providing any support for these external
libraries.
– again, my recommendation is to stick with the inefficient MATLAB
while you learn about computer vision.
19 Monday, January 11, 2010 ...
View
Full Document
 Fall '08
 Staff
 Computer vision, Weight function, Feature detection, Blob detection, corner detection, Correspondence problem

Click to edit the document details