EECE253_07_Convolution

EECE253_07_Convolution - EECE\CS 253 Image Processing...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: EECE\CS 253 Image Processing Lecture Notes: Spatial Convolution Lecture Notes Richard Alan Peters II Department of Electrical Engineering and Computer Science Fall Semester 2011 This work is licensed under the Creative Commons Attribution-Noncommercial 2.5 License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc/2.5/ or send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA. Spatial Filtering Let I and J be images such that J = T [I]. T[·] represents a transformation, such that, J (r , c) = T [ I ](r , c) = f ({ I (r, c) r Î {r - s,..., r ,...r + s} , c Î {c - d ,..., c,...c + d } }). That is, the value of the transformed image, J, at pixel location (r,c) is a function of the values of the original image, I, in a 2s +1 × 2d +1 rectangular neighborhood centered on pixel location (r,c). 2011-09-29 1999-2011 by Richard Alan Peters II 2 Moving Windows The value, J(r,c) = T[I](r,c), is a function of a rectangular neighborhood centered on pixel location (r,c) in I. There is a different neighborhood for each pixel location, but if the dimensions of the neighborhood are the same for each location, then transform T is sometimes called a moving window transform. 2011-09-29 1999-2011 by Richard Alan Peters II 3 Moving-Window Transformations Neutral Buoyancy Facility at NASA Johnson Space Center We’ll take a section of this image to demonstrate the MWT photo: R.A.Peters II, 1999 2011-09-29 1999-2011 by Richard Alan Peters II 4 Moving-Window Transformations operate on this region 2011-09-29 1999-2011 by Richard Alan Peters II 5 Pixelize the section to better see the effects. Moving-Window Transformations apply a pixel grid 2011-09-29 1999-2011 by Richard Alan Peters II 6 Pixelize the section to better see the effects. Moving-Window Transformations sample (average in the squares). 2011-09-29 1999-2011 by Richard Alan Peters II 7 Moving-Window Transformations lets get some perspective on this 2011-09-29 1999-2011 by Richard Alan Peters II 8 Moving-Window Transformations a neighborhood defined by a weight matrix 2011-09-29 1999-2011 by Richard Alan Peters II 9 Moving-Window Transformations neighborhoods at other pixel locations 2011-09-29 1999-2011 by Richard Alan Peters II 10 Linear Moving-Window Transformations ( i.e. convolution) The output of the transform at each pixel is the (weighted) average of the pixels in the neighborhood. 2011-09-29 1999-2011 by Richard Alan Peters II 11 Moving-Window Transformations result of a 9 x 9 uniform averaging 2011-09-29 1999-2011 by Richard Alan Peters II 12 Convolution: Mathematical Representation If a MW transformation is linear then it is a convolution: ¥¥ J (r , c) = [ I * h ] (r , c) = ò ò I(r - r, c - c)h(r, c) dr d c , -¥ -¥ for a real image (I:×→), or for a digital image (I:×→): J (r , c) = [ I * h ] (r , c) = s d å å B ( r - r , c - c) h (r , c) r =- s c=- d 2011-09-29 1999-2011 by Richard Alan Peters II 13 Convolution Mask (Weight Matrix) • The object, h(ρ,χ), in the equation is a weighting function, or in the discrete case, a rectangular matrix of numbers. • The matrix is the moving window. • Pixel (r,c) in the output image is the weighted sum of pixels from the original image in the neighborhood of (r,c) traced by the matrix. • Each pixel in the neighborhood of (r,c) is multiplied by the corresponding matrix value — after the matrix is rotated by 180º. (See slide 22). • The sum of those products is the value of pixel (r,c) in the output image 2011-09-29 1999-2011 by Richard Alan Peters II 14 Convolution Masks: Moving Window mask origin rotate 180°… translate to pixel loc (r,c) 2011-09-29 … around pixel loc (r,c) 1999-2011 by Richard Alan Peters II 15 Convolution Masks: Moving Window multiplies pixel I(r-1,c-2) multiplies pixel I(r,c-1) 2011-09-29 1999-2011 by Richard Alan Peters II multiplies pixel I(r-1,c) multiplies pixel I(r+1,c+1) 16 Convolution by Moving Window c 2011-09-29 f i g 1999-2011 by Richard Alan Peters II abc def ghi 17 Another example Moving Window Transform: Example original 2011-09-29 3x3 average 1999-2011 by Richard Alan Peters II 18 Moving Window Transform: Example original 2011-09-29 3x3 average 1999-2011 by Richard Alan Peters II 19 Moving Window Transform: Example original 2011-09-29 3x3 average 1999-2011 by Richard Alan Peters II 20 Moving Window Transform: Example original 2011-09-29 3x3 average 1999-2011 by Richard Alan Peters II 21 Moving Window Transform: Example original 2011-09-29 3x3 average 1999-2011 by Richard Alan Peters II 22 Moving Window Transform: Example original 2011-09-29 3x3 average 1999-2011 by Richard Alan Peters II 23 Moving Window Transform: Example original 2011-09-29 3x3 average 1999-2011 by Richard Alan Peters II 24 Convolution by Rotating and Shifting the Weight Matrix At the locations not shown, the results were zeros. Shifted weight matrix Result of sum of products Location of impulse Accumulated nonzero results The original image has a black impulse at the center and zeros (white) elsewhere. 2011-09-29 The weight matrix has a gray ‘L’ at its left and zeros (white) elsewhere. 1999-2011 by Richard Alan Peters II The resulting image has a copy of the weight matrix pegged to the impulse location. 14 25 Symmetric Weight Matrix A symmetric weight matrix is unchanged by rotation through 180°. 2011-09-29 1999-2011 by Richard Alan Peters II 26 Three ways to compute a convolution 1. Moving window transform as just shown. 2. Shift multiply add. 3. Fourier transform. 1. 2011-09-29 2. 1999-2011 by Richard Alan Peters II 3. 27 Shift-Multiply-Add Approach • The image is copied 1 time for each element in the convolution mask. • Each copy is shifted relative to the original by the displacement of its associated mask element. • Each copy is multiplied by the value of its associated mask element. • The set of shifted and multiplied images is summed pixel wise. 2011-09-29 1999-2011 by Richard Alan Peters II 28 Convolution by an Impulse An impulse is a digital image, that has a single pixel with value 1; all others have value zero. An impulse at location (ρ, χ) is represented by: d ( r - r , c - c) = { 1, if r = r and c = c 0, otherwise If an image is convolved with an impulse of weight w at location (ρ, χ), then the image is multiplied by w and shifted in location down by ρ pixels and to the right by χ pixels. [I * wd (r - r, c - c)] (r , c) = wI (r - r, c - c). 2011-09-29 1999-2011 by Richard Alan Peters II 29 Convolution by an Impulse δ (r − 16, c − 16) Shifted down and to the right by 16 pixels. 2011-09-29 1999-2011 by Richard Alan Peters II 30 Convolution by Two Impulses 1 2 d (r - 0, c - 0) 1 2 d (r -16, c -16) Two copies, one moved, one not moved, averaged. 2011-09-29 1999-2011 by Richard Alan Peters II 31 Convolution by Three Impulses 1 3 d (r + 16, c + 16) 1 3 d (r - 0, c - 0) 1 3 d (r -16, c -16) Three copies, two moved, Weights averaged. one not moved, = 1/3 2011-09-29 1999-2011 by Richard Alan Peters II 32 Convolution by Five Impulses 1 5 d (r + 16, c + 16) 1 5 1 5 1 5 d (r + 16, c -16) d (r - 0, c - 0) d (r -16, c + 16) 1 5 d (r -16, c -16) Five copies, four moved, one not moved, averaged. 2011-09-29 1999-2011 by Richard Alan Peters II 33 Convolution by Five Impulses Moved adjacent to each other, the convolution becomes a blurring filter. 2011-09-29 1999-2011 by Richard Alan Peters II 34 Convolution by Five Impulses The impulses become values in a 3x3 neighborhood. 2011-09-29 1999-2011 by Richard Alan Peters II 35 Convolution by Five Impulses The convolution mask has five elements at 1/5 and four at 0. 2011-09-29 1999-2011 by Richard Alan Peters II 36 Convolution by Copying, Multiplying, and Shifting the Image For each element h(rh,ch) in weight matrix, h, image I is copied into a zero-padded image, P, starting at (rh,ch). Each P is multiplied by the corresponding weight, h(rh,ch). All the P images are summed pixel-wise then divided by the sum of the elements of h. The result is cropped out of the center of the accumulated Ps. original image, I padded image, P effective neighborhood weight matrix aligned pixels to be summed weight for image 2011-09-29 1999-2011 by Richard Alan Peters II 37 Convolution by Copying, Multiplying, and Shifting the Image The original image has a black impulse at the center and zeros (white) elsewhere. The weight matrix has a gray ‘L’ at its left and zeros (white) elsewhere. The resulting image has a copy of the weight matrix pegged to the impulse location. original image, I effective neighborhood padded image, P 2011-09-29 In the result, the origin of the weight matrix coincides with the original location of the impulse. 1999-2011 by Richard Alan Peters II 38 Convolution by Copying, Multiplying, and Shifting the Image Each copy of the (entire) image is multiplied by the value of the weight matrix in black square (here, white = 0) before being accumulated (pixelwise) in the padded image The position of the black square relative the center of the weight matrix indicates the shift of the original image relative to the middle of the padded image. 2011-09-29 In this image, only the pixel in the center is nonzero so only it shows a result when the image is multiplied by a nonzero value 1999-2011 by Richard Alan Peters II 39 Convolution by Copying, Multiplying, and Shifting the Image Each copy of the (entire) image is multiplied by the value of the weight matrix in black square (here, white = 0) before being accumulated (pixelwise) in the padded image The position of the black square relative the center of the weight matrix indicates the shift of the original image relative to the middle of the padded image. 2011-09-29 In this image, only the pixel in the center is nonzero so only it shows a result when the image is multiplied by a nonzero value 1999-2011 by Richard Alan Peters II 40 Zero Padding an Image for Convolution: Variable Names. weight matrix, h n C cpad d rpad B m R cpad = rpad = hcorig hrorig 2011-09-29 floor( floor( = cpad = rpad n m + + Image, I /2) /2) 1 1 1999-2011 by Richard Alan Peters II 41 To use the image shift-multiplyaccumulate algorithm, create an accumulator image, A, that is R+m-1 rows by C+n-1 columns 2011-09-29 Image I is RxC C+n-1 A I R+m-1 Convolution by Copying and Shifting the Image R n C 1999-2011 by Richard Alan Peters II m h 42 Convolution by Copying, Multiplying, and Shifting the Image 13x13 image convolved by 6x6 mask. accumulator output image Image is constant; mask has only 6 nonzero values all on the diagonal. conv. mask Image is shifted to mask location, multiplied by value, and accumulated. 2011-09-29 1999-2011 by Richard Alan Peters II 43 hcorig hcorig+C-1 hrorig Convolution by Copying and Shifting the Image When done, copy the output image from the accumulator starting at (hrorig, hcorig) and ending at (hrorig+R-1, hcorig+C-1) 2011-09-29 hrorig+R-1 output image, J 1999-2011 by Richard Alan Peters II R C 44 Convolution Examples: Original Images 2011-09-29 1999-2011 by Richard Alan Peters II 45 111 1 111 9 111 Convolution Examples: 3×3 Blur 2011-09-29 1999-2011 by Richard Alan Peters II 46 11111 11111 1 11111 25 1 1 1 1 1 11111 Convolution Examples: 5×5 Blur 2011-09-29 1999-2011 by Richard Alan Peters II 47 111111111 111111111 111111111 111111111 1 111111111 81 1 1 1 1 1 1 1 1 1 111111111 111111111 111111111 2011-09-29 Convolution Examples: 9×9 Blur 1999-2011 by Richard Alan Peters II 48 11111111111111111 11111111111111111 11111111111111111 11111111111111111 11111111111111111 11111111111111111 11111111111111111 11111111111111111 1 11111111111111111 289 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11111111111111111 11111111111111111 11111111111111111 11111111111111111 11111111111111111 11111111111111111 11111111111111111 2011-09-29 Convolution Examples: 17×17 Blur 1999-2011 by Richard Alan Peters II 49 Vertical Edge Detection 255 Image r0 Backward Difference r0 Forward Difference r0 Sum of Differences c r0 2011-09-29 0 -255 c c c 1999-2011 by Richard Alan Peters II 50 Symmetric Edge Detection 510 255 0 -255 2011-09-29 -1 2 -1 -1 2 -1 1999-2011 by Richard Alan Peters II -1 -1 4 -1 -1 51 Convolution Examples: Original Images 2011-09-29 1999-2011 by Richard Alan Peters II 52 −1 2 −1 Convolution Examples: Vertical Difference 2011-09-29 1999-2011 by Richard Alan Peters II 53 [−1 2 − 1] Convolution Examples: Horizontal Difference 2011-09-29 1999-2011 by Richard Alan Peters II 54 0 −1 0 −1 4 −1 0 −1 0 Convolution Examples: H + V Diff. 2011-09-29 1999-2011 by Richard Alan Peters II 55 −1 0 0 020 0 0 −1 Convolution Examples: Diagonal Difference 2011-09-29 1999-2011 by Richard Alan Peters II 56 0 0 −1 020 −1 0 0 Convolution Examples: Diagonal Difference 2011-09-29 1999-2011 by Richard Alan Peters II 57 −1 0 −1 040 −1 0 −1 Convolution Examples: D + D Difference 2011-09-29 1999-2011 by Richard Alan Peters II 58 −1 −1 −1 −1 8 −1 −1 −1 −1 Convolution Examples: H + V + D Diff. 2011-09-29 1999-2011 by Richard Alan Peters II 59 Convolution Examples: Original Images 2011-09-29 1999-2011 by Richard Alan Peters II 60 ...
View Full Document

This note was uploaded on 12/06/2011 for the course EECE 253 taught by Professor Alanpeters during the Summer '07 term at Vanderbilt.

Ask a homework question - tutors are online