ch04 - Ch 4 Feature Detection •  Overview • ...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Ch 4 - Feature Detection •  Overview •  Feature Detection –  Intensity Extrema –  Blob Detection –  Corner Detection •  Feature Descriptors •  Feature Matching •  Conclusion Overview •  Goal of feature detection is to find geometric objects in an image that are visually interesting 4.1 Points and patches 209 boring interesting Figure 4.3 Image pairs with extracted patches below. Notice how some patches can be localized or matched with higher accuracy than others. Overview •  Features typically have descriptors that capture local geometric properties in the image •  Features descriptors can be used for a wide range of computer vision applications: –  Align multiple images to each other –  Track the motion of objects in an image sequence –  Perform object or face recognition –  Detect defects in manufactured objects 206 Computer Vision: Algorithms and Applications (August 18, 2010 draft) Feature Detection •  Features can be computed from geometric or statistical properties of an image •  The most common types of image features are points, lines, or regions 206 Computer Vision: Algorithms and Applications (August 18, 2010 draft) 206 Computer Vision: Algorithms and Applications (August 18, 2010 draft) (a) (b) (c) (a) (b) (a) (d) Figure 4.1 A variety of feature detectors and descriptors can be used to analyze, describe and c match images: (a) point-like interest operators (Brown, Szeliski, and Winder 2005) ￿ 2005 c IEEE; (b) region-like interest operators (Matas, Chum, Urban et al. 2004) ￿ 2004 Elsevier; c (c) edges (Elder and Goldberg 2001) ￿ 2001 IEEE; (d) straight lines (Sinha, Steedly, Szeliski (b) Feature Detection •  In order to be useful features should be invariant to image translation and rotation –  Rotating or translating an image will yield same number of features but in different locations •  In some cases features may be scale invariant –  Resizing or blurring an image will yield a subset of the original image features corresponding to the larger scale geometric objects Feature Detection •  Points are the most basic geometric feature to detect in an image but finding interesting points is not easy •  We look at small neighborhoods in the image to find points that stand out in some way –  Intensity extrema (maxima and minima) –  Find blobs using differential properties –  Find corners using statistical properties Intensity Extrema Intensity maxima = positions of stars or galaxies in a space image Intensity Extrema Intensity minima = positions of eyes, nose or mouth an image of face Intensity Extrema •  How do we locate intensity extrema? –  Scan image top-bottom and left-right –  Look in NxN neighborhood of each pixel(x,y) –  Intensity maxima if pixel(x,y) > all others –  Intensity minima if pixel(x,y) < all others 8 10 9 11 10 11 9 10 11 12 11 12 10 12 15 10 9 10 9 11 13 12 10 11 8 9 10 10 11 12 •  15 is maxima in 3x3 region •  9 is minima in 3x3 region Intensity Extrema for (int y = 1; y < Ydim - 1; y++) for (int x = 1; x < Xdim - 1; x++) { // Check for maximum PIXEL pixel = Data2D[y][x]; if ((pixel > Data2D[y - 1][x - 1]) && (pixel > Data2D[y - 1][x]) && (pixel > Data2D[y - 1][x + 1]) && (pixel > Data2D[y][x - 1]) && (pixel > Data2D[y][x + 1]) && (pixel > Data2D[y + 1][x - 1]) && (pixel > Data2D[y + 1][x]) && (pixel > Data2D[y + 1][x + 1])) out.Data2D[y][x] = 1; Intensity Extrema // Check for minimum if ((pixel < Data2D[y - 1][x - 1]) && (pixel < Data2D[y - 1][x]) && (pixel < Data2D[y - 1][x + 1]) && (pixel < Data2D[y][x - 1]) && (pixel < Data2D[y][x + 1]) && (pixel < Data2D[y + 1][x - 1]) && (pixel < Data2D[y + 1][x]) && (pixel < Data2D[y + 1][x + 1])) out.Data2D[y][x] = -1; } Intensity Extrema •  What about pixels with same intensity value? –  Can say maxima if pixel(x,y) >= all others –  Can say minima if pixel(x,y) <= all others –  Often results in extrema regions instead of points 9 10 10 12 11 11 9 10 11 12 11 12 10 12 15 15 11 11 9 11 15 12 11 11 9 9 10 10 11 12 •  All points value 15 are maxima in corresponding 3x3 regions •  All points value 9 are minima in corresponding 3x3 regions Intensity Extrema •  How can we find important extrema? –  Reduce the number of intensity extrema –  Use larger neighborhood when scanning image or perform Gaussian blurring before scanning –  Size of neighborhood N or blurring standard deviation σ will define the scale of the extrema –  Can also look at multiple N or σ values to obtain scale invariant feature points Intensity Extrema σ=1 Intensity Extrema σ=2 Intensity Extrema σ=4 Intensity Extrema σ=8 Blob Detection •  The goal of blob detection is to locate regions of an image that are visibly lighter or darker than their surrounding regions •  One way to detect blobs is to –  Look at NxN regions in an image and calculate average intensities inside / outside radius R –  Light blob: average inside >> average outside –  Dark blob: average inside << average outside Blob Detection •  Another approach would be to convolve image with black / white circle masks with radius R •  Center of light blobs - peaks of white mask •  Center of dark blobs - peaks of black mask •  Vary radius R to find different size blobs Blob Detection •  Better approach is to filter image with Laplacian of Gaussian (LoG) mask –  Peaks and pits of LoG image mark centers of blobs –  Size of blobs given by sigma of Gaussian Blob Detection Blob Detection Blob Detection Blob Detection Blob Detection •  The LoG filter can be approximated using Difference of Gaussian (DoG) filters –  Easy to implement in multiscale applications that are already using Gaussian smoothing •  Determinant of Hessian (DoH) can also be used for blob detection in place of LoG –  Ixx . Iyy – Ixy2 is also rotationally invariant –  Does not infringe on SIFTs patent of LoG for multiscale blob detection Corner Detection •  A corner can be defined as the intersection of two edges in an image or a point where there are two different dominant edge directions Corner Detection •  Corner detectors are typically based on local statistical or differential properties of an image from: Corner Detection •  Moravec (1977) –  Developed to help navigate the Stanford cart –  Sum of square differences SSD is used to measure similarity of overlapping patches in the image –  Corner strength is smallest SSD between patch and its 8 neighbors (N,S,E,W,NE,NW,SE,SW) –  Corner is present in an image when corner strength is locally maximal and above a threshold Corner Detection •  SSD calculation: Corner Detection •  Different local neighborhoods for SSD calculation give different corner responses Interior Edge Corner Point Corner Detection •  Corners on block image Corner Detection •  Corners on house image Feature Descriptors •  Feature detection locates (x,y) points in an image that are interesting in some way –  These (x,y) locations by themselves are not very useful for computer vision applications •  We use feature descriptors to capture what makes these points interesting –  We want local geometric properties that are invariant to translation, rotation, and scaling Feature Descriptors •  What local geometric information can we obtain from a point in an image? –  We can look at differential properties –  We can look at statistical properties •  The most common approach is to look at the image gradient near feature points •  Gradient magnitude = (Ix2 + Iy2)1/2 •  Gradient direction = arctan(Iy / Ix) Feature Descriptors •  Calculate gradient magnitude and direction in a small region around feature point •  Quantize angles to 8 directions and calculate 4.1 Points and patches weighted angle histogram to get 8 features 219 Figure 4.12 A dominant orientation estimate can be computed by creating a histogram of Feature Descriptors 219 219 Figure 4.12 A dominant orientation estimate can be computed by creating a histogram of all the gradient orientations (weighted by their magnitudes or after thresholding out small c gradients) and then finding the significant peaks in this distribution (Lowe 2004) ￿ 2004 Springer. A better method is to estimate a dominant orientation at the local orientation and scale of a keypoint have been estimat around the detected point can be extracted and used to form a f and 4.17). The simplest possible orientation estimate is the average g the keypoint. If a Gaussian weighting function is used (Brow this average gradient is equivalent to a first-order steerable filte computed using an image convolution with the horizontal an Figure 4.12 A dominant orientation estimate can be compu all the gradient orientations (weighted by their magnitudes o gradients) and then finding the significant peaks in this distr Springer. 4.1 Points and patches •  The angle histogram is invariant to translation 4.1 Points and patches •  What about image rotation? Feature Descriptors •  How can we fix this problem? –  Find the dominant gradient direction –  Rotate the region so the dominant gradient direction is at angle zero –  Recalculate the angle histogram •  We can get almost the same result by shifting the angle histogram to put largest value into the angle zero bucket Feature Descriptors •  More geometric information can be obtained by calculating multiple angle histograms 224 Computer Vision: Algorithms and Applications (August 18, 2010 draft) 4x8=32 features 4.1 Points and patches (a) image gradients 225 (b) keypoint descriptor Figure 4.18 A schematic representation of Lowe’s (2004) scale invariant feature transform (SIFT): (a) Gradient orientations and magnitudes are computed at each pixel and weighted by a Gaussian fall-off function (blue circle). (b) A weighted gradient orientation histogram is then computed in each subregion, using trilinear interpolation. While this figure shows an 8 × 8 pixel patch and a 2 × 2 descriptor array, Lowe’s actual implementation uses 16 × 16 patches and a 4 × 4 array of eight-bin histograms. 11, and 15, with eight angular bins (except for the central region), for a total of 17 spa(a) image gradients (b) keypoint descriptor tial bins and 16 orientation bins. The 272-dimensional histogram is then projected onto a 128-dimensional descriptor using PCA trained on a large database. In their evaluation, Figure 4.19 The gradient location-orientation histogram (GLOH) descriptor uses log-polar Mikolajczyk and Schmid (2005) found that GLOH, which has the best performance overall, bins instead of square bins to compute orientation histograms (Mikolajczyk and Schmid outperforms SIFT by a small margin. 17x8=136 features Feature Descriptors •  How many feature values are best? –  More feature values => Increase descriptive power, space required and comparison time –  Fewer feature values => Decrease descriptive power, space required and comparison time •  Answer depends on needs of CV application –  SIFT uses 4x4x8=128 features –  PCA-SIFT uses principal component analysis to reduce 3042 raw features to 20-32 features Feature Matching •  There are two issues in feature matching –  Matching strategy - how feature vectors are compared to each other –  Data structures and algorithms - how feature vectors are stored and retrieved •  We must handle lots of data quickly and get robust and accurate feature matching results Feature Matching •  The easiest way to compare feature vectors is to calculate Euclidean distance between them 1/ 2 # 2& A ! B = % " ( Ai ! Bi ) ( $ i=1.. N ' •  Since square roots are slow some applications compare squared distances or they calculate sum absolute differences (L1 norm) Feature Matching •  Another approach is to normalize the feature vectors and calculate the cosine of the angle between feature vectors ˆ A= A/ A ˆ B=B/ B ˆˆ cos! = A • B = ˆˆ " A !B i i i=1.. N •  When cosθ = 1 feature vectors match •  When cosθ = 0 feature vectors do not match Feature Matching •  Feature vectors can also be normalized so the dynamic range in each dimension is the same min i = min !i ( Ai ) max i = max !i ( Ai ) ! Ai = Ai / (max i " min i ) •  This way each dimension will have the same weight in the difference calculation Feature Matching •  Another approach is to normalize vectors based on the standard deviation of each dimension and calculate the Mahalonobis distance between feature vectors µi = " Ai / M !i 2 ! = " ( Ai # µi ) / M 2 i !i 1/ 2 ! Ai = Ai / ! i $ 2' !! !! A # B = & " Ai # Bi ) % i=1.. N ( ( ) Feature Matching •  The most basic data structure for feature vectors is a dynamic 2D array –  Trivial to insert or remove data –  Requires O(N) search to match vectors Feature Matching •  A binary tree can be used to quickly insert, delete and search 1D data •  Trees can be generalized to 2D or 3D data values by having 4 or 8 children per node Quadtree Octree Feature Matching •  Simply extending the quadtree/octree concept to more dimensions wastes a lot of space •  The K-D tree solves this by creating a binary space partition (BSP) of the data Feature Matching •  There are many variations on K-D trees with different BSPs and different search algorithms –  Best solutions are O(logN) •  Finally, a number of hash table techniques have been devised to store/search features –  Searching is approximate instead of exact –  Best solutions are almost constant time Conclusions •  Goal of feature detection is to find geometric objects in an image that are visually interesting –  These features can then be used to create CV applications for image alignment, object tracking and recognition, and defect detection •  Feature detection and matching is a large area –  Different feature detection methods –  Different feature descriptors –  Different feature matching techniques ...
View Full Document

{[ snackBarMessage ]}

Ask a homework question - tutors are online