This preview shows page 1. Sign up to view the full content.
Unformatted text preview: EECE\CS 253 Image Processing
Lecture Notes: Spatial Convolution
Lecture Notes Richard Alan Peters II
Department of Electrical Engineering and
Computer Science
Fall Semester 2011 This work is licensed under the Creative Commons AttributionNoncommercial 2.5 License. To view a copy of this license, visit http://creativecommons.org/licenses/bync/2.5/ or
send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA. Spatial Filtering
Let I and J be images such that J = T [I].
T[·] represents a transformation, such that,
J (r , c) = T [ I ](r , c) =
f ({ I (r, c) r Î {r  s,..., r ,...r + s} , c Î {c  d ,..., c,...c + d } }). That is, the value of the transformed image, J, at pixel location
(r,c) is a function of the values of the original image, I, in a 2s +1
× 2d +1 rectangular neighborhood centered on pixel location (r,c).
20110929 19992011 by Richard Alan Peters II 2 Moving Windows The value, J(r,c) = T[I](r,c), is a function of a
rectangular neighborhood centered on pixel
location (r,c) in I.
There is a different neighborhood for each pixel
location, but if the dimensions of the neighborhood are the same for each location, then transform T is sometimes called a moving window
transform. 20110929 19992011 by Richard Alan Peters II 3 MovingWindow Transformations Neutral
Buoyancy
Facility at
NASA
Johnson
Space
Center We’ll take a
section of
this image to
demonstrate
the MWT photo: R.A.Peters II, 1999 20110929 19992011 by Richard Alan Peters II 4 MovingWindow Transformations operate on this region 20110929 19992011 by Richard Alan Peters II 5 Pixelize the section to
better see the effects. MovingWindow Transformations apply a pixel grid 20110929 19992011 by Richard Alan Peters II 6 Pixelize the section to
better see the effects. MovingWindow Transformations sample (average
in the squares).
20110929 19992011 by Richard Alan Peters II 7 MovingWindow Transformations lets get
some
perspective
on this
20110929 19992011 by Richard Alan Peters II 8 MovingWindow Transformations a neighborhood defined
by a weight matrix
20110929 19992011 by Richard Alan Peters II 9 MovingWindow Transformations neighborhoods at other pixel locations
20110929 19992011 by Richard Alan Peters II 10 Linear MovingWindow
Transformations
( i.e. convolution) The output of the
transform at each pixel
is the (weighted)
average of the pixels in
the neighborhood. 20110929 19992011 by Richard Alan Peters II 11 MovingWindow Transformations result of a 9 x 9
uniform averaging 20110929 19992011 by Richard Alan Peters II 12 Convolution: Mathematical Representation
If a MW transformation is linear then it is a convolution:
¥¥ J (r , c) = [ I * h ] (r , c) = ò ò I(r  r, c  c)h(r, c) dr d c , ¥ ¥ for a real image (I:×→), or for a digital image (I:×→):
J (r , c) = [ I * h ] (r , c) = s d å å B ( r  r , c  c) h (r , c) r = s c= d 20110929 19992011 by Richard Alan Peters II 13 Convolution Mask (Weight Matrix)
• The object, h(ρ,χ), in the equation is a weighting function, or in
the discrete case, a rectangular matrix of numbers.
• The matrix is the moving window.
• Pixel (r,c) in the output image is the weighted sum of pixels
from the original image in the neighborhood of (r,c) traced by
the matrix.
• Each pixel in the neighborhood of (r,c) is multiplied by the
corresponding matrix value — after the matrix is rotated by
180º. (See slide 22).
• The sum of those products is the value of pixel (r,c) in the
output image
20110929 19992011 by Richard Alan Peters II 14 Convolution Masks: Moving Window mask
origin rotate 180°… translate
to pixel
loc (r,c)
20110929 … around
pixel loc
(r,c)
19992011 by Richard Alan Peters II 15 Convolution Masks: Moving Window
multiplies
pixel
I(r1,c2) multiplies
pixel
I(r,c1) 20110929 19992011 by Richard Alan Peters II multiplies
pixel
I(r1,c) multiplies
pixel
I(r+1,c+1)
16 Convolution by Moving Window c 20110929 f i g 19992011 by Richard Alan Peters II abc
def
ghi 17 Another example Moving Window Transform: Example original
20110929 3x3 average
19992011 by Richard Alan Peters II 18 Moving Window Transform: Example original
20110929 3x3 average
19992011 by Richard Alan Peters II 19 Moving Window Transform: Example original
20110929 3x3 average
19992011 by Richard Alan Peters II 20 Moving Window Transform: Example original
20110929 3x3 average
19992011 by Richard Alan Peters II 21 Moving Window Transform: Example original
20110929 3x3 average
19992011 by Richard Alan Peters II 22 Moving Window Transform: Example original
20110929 3x3 average
19992011 by Richard Alan Peters II 23 Moving Window Transform: Example original
20110929 3x3 average
19992011 by Richard Alan Peters II 24 Convolution by Rotating and Shifting the Weight Matrix
At the locations
not shown, the
results were
zeros. Shifted weight
matrix
Result of sum
of products Location of
impulse Accumulated
nonzero results The original image has a
black impulse at the center
and zeros (white) elsewhere. 20110929 The weight matrix has a
gray ‘L’ at its left and zeros
(white) elsewhere. 19992011 by Richard Alan Peters II The resulting image has a copy
of the weight matrix pegged
to the impulse location. 14 25 Symmetric Weight Matrix A symmetric weight matrix is unchanged by rotation through 180°. 20110929 19992011 by Richard Alan Peters II 26 Three ways to compute a convolution
1. Moving window transform as just shown.
2. Shift multiply add.
3. Fourier transform. 1. 20110929 2. 19992011 by Richard Alan Peters II 3. 27 ShiftMultiplyAdd Approach
• The image is copied 1 time for each element in
the convolution mask.
• Each copy is shifted relative to the original by
the displacement of its associated mask element.
• Each copy is multiplied by the value of its
associated mask element.
• The set of shifted and multiplied images is
summed pixel wise. 20110929 19992011 by Richard Alan Peters II 28 Convolution by an Impulse
An impulse is a digital image, that has a single pixel with
value 1; all others have value zero. An impulse at location
(ρ, χ) is represented by:
d ( r  r , c  c) = { 1, if r = r and c = c
0, otherwise If an image is convolved with an impulse of weight w at
location (ρ, χ), then the image is multiplied by w and shifted
in location down by ρ pixels and to the right by χ pixels. [I * wd (r  r, c  c)] (r , c) = wI (r  r, c  c).
20110929 19992011 by Richard Alan Peters II 29 Convolution by an Impulse δ (r − 16, c − 16) Shifted down and to
the right by 16 pixels.
20110929 19992011 by Richard Alan Peters II 30 Convolution by Two Impulses 1
2 d (r  0, c  0) 1
2 d (r 16, c 16) Two copies, one moved,
one not moved, averaged.
20110929 19992011 by Richard Alan Peters II 31 Convolution by Three Impulses
1
3 d (r + 16, c + 16) 1
3 d (r  0, c  0) 1
3 d (r 16, c 16) Three copies, two moved,
Weights averaged.
one not moved, = 1/3
20110929 19992011 by Richard Alan Peters II 32 Convolution by Five Impulses
1
5 d (r + 16, c + 16) 1
5 1
5 1
5 d (r + 16, c 16) d (r  0, c  0) d (r 16, c + 16) 1
5 d (r 16, c 16) Five copies, four moved,
one not moved, averaged.
20110929 19992011 by Richard Alan Peters II 33 Convolution by Five Impulses Moved adjacent to each
other, the convolution
becomes a blurring filter. 20110929 19992011 by Richard Alan Peters II 34 Convolution by Five Impulses The impulses become values
in a 3x3 neighborhood. 20110929 19992011 by Richard Alan Peters II 35 Convolution by Five Impulses The convolution mask
has five elements at
1/5 and four at 0. 20110929 19992011 by Richard Alan Peters II 36 Convolution by Copying, Multiplying, and Shifting the Image
For each element h(rh,ch)
in weight matrix, h,
image I is copied into a
zeropadded image, P,
starting at (rh,ch).
Each P is multiplied by
the corresponding weight,
h(rh,ch).
All the P images are
summed pixelwise then
divided by the sum of the
elements of h. The result
is cropped out of the
center of the accumulated
Ps. original image, I
padded image, P
effective neighborhood
weight matrix
aligned pixels to
be summed
weight for image 20110929 19992011 by Richard Alan Peters II 37 Convolution by Copying, Multiplying, and Shifting the Image
The original image has a
black impulse at the center
and zeros (white) elsewhere. The weight matrix has a
gray ‘L’ at its left and zeros
(white) elsewhere. The resulting image has a
copy of the weight matrix
pegged to the impulse
location. original image, I
effective neighborhood
padded image, P
20110929 In the result, the origin of the
weight matrix coincides with the
original location of the impulse. 19992011 by Richard Alan Peters II 38 Convolution by
Copying, Multiplying,
and Shifting the Image Each copy of the (entire) image is multiplied by the value
of the weight matrix in black square (here, white = 0)
before being accumulated (pixelwise) in the padded image The position of
the black
square relative
the center of
the weight
matrix
indicates the
shift of the
original image
relative to the
middle of the
padded image. 20110929 In this
image, only
the pixel in
the center is
nonzero so
only it shows
a result when
the image is
multiplied by
a nonzero
value 19992011 by Richard Alan Peters II 39 Convolution by
Copying, Multiplying,
and Shifting the Image Each copy of the (entire) image is multiplied by the value
of the weight matrix in black square (here, white = 0)
before being accumulated (pixelwise) in the padded image The position of
the black
square relative
the center of
the weight
matrix
indicates the
shift of the
original image
relative to the
middle of the
padded image. 20110929 In this
image, only
the pixel in
the center is
nonzero so
only it shows
a result when
the image is
multiplied by
a nonzero
value 19992011 by Richard Alan Peters II 40 Zero Padding an Image for
Convolution: Variable Names. weight matrix, h n C cpad d rpad B m
R
cpad =
rpad =
hcorig
hrorig 20110929 floor(
floor(
= cpad
= rpad n
m
+
+ Image, I /2)
/2)
1
1 19992011 by Richard Alan Peters II 41 To use the image
shiftmultiplyaccumulate
algorithm, create
an accumulator
image, A, that is
R+m1 rows by
C+n1 columns 20110929 Image I is RxC C+n1 A
I R+m1 Convolution
by Copying
and Shifting
the Image R
n C 19992011 by Richard Alan Peters II m h 42 Convolution by Copying, Multiplying, and Shifting the Image
13x13 image
convolved by
6x6 mask. accumulator
output image Image is constant;
mask has only 6
nonzero values all
on the diagonal. conv. mask Image is shifted to
mask location,
multiplied by value,
and accumulated.
20110929 19992011 by Richard Alan Peters II 43 hcorig hcorig+C1 hrorig Convolution
by Copying
and Shifting
the Image When done, copy the
output image from the
accumulator starting
at (hrorig, hcorig) and
ending at (hrorig+R1,
hcorig+C1)
20110929 hrorig+R1 output image, J 19992011 by Richard Alan Peters II R C
44 Convolution Examples: Original Images 20110929 19992011 by Richard Alan Peters II 45 111
1
111
9 111 Convolution Examples: 3×3 Blur 20110929 19992011 by Richard Alan Peters II 46 11111
11111
1
11111
25 1 1 1 1 1
11111 Convolution Examples: 5×5 Blur 20110929 19992011 by Richard Alan Peters II 47 111111111
111111111
111111111
111111111
1
111111111
81 1 1 1 1 1 1 1 1 1
111111111
111111111
111111111 20110929 Convolution Examples: 9×9 Blur 19992011 by Richard Alan Peters II 48 11111111111111111
11111111111111111
11111111111111111
11111111111111111
11111111111111111
11111111111111111
11111111111111111
11111111111111111
1
11111111111111111
289 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
11111111111111111
11111111111111111
11111111111111111
11111111111111111
11111111111111111
11111111111111111
11111111111111111 20110929 Convolution Examples: 17×17 Blur 19992011 by Richard Alan Peters II 49 Vertical Edge Detection
255 Image r0 Backward
Difference r0 Forward
Difference r0 Sum of
Differences c r0 20110929 0
255 c c c
19992011 by Richard Alan Peters II 50 Symmetric Edge Detection 510
255
0
255 20110929 1 2 1 1
2
1 19992011 by Richard Alan Peters II 1
1 4 1
1 51 Convolution Examples: Original Images 20110929 19992011 by Richard Alan Peters II 52 −1
2
−1 Convolution Examples: Vertical Difference 20110929 19992011 by Richard Alan Peters II 53 [−1 2 − 1] Convolution Examples: Horizontal Difference 20110929 19992011 by Richard Alan Peters II 54 0 −1 0
−1 4 −1
0 −1 0 Convolution Examples: H + V Diff. 20110929 19992011 by Richard Alan Peters II 55 −1 0 0
020
0 0 −1 Convolution Examples: Diagonal Difference 20110929 19992011 by Richard Alan Peters II 56 0 0 −1
020
−1 0 0 Convolution Examples: Diagonal Difference 20110929 19992011 by Richard Alan Peters II 57 −1 0 −1
040
−1 0 −1 Convolution Examples: D + D Difference 20110929 19992011 by Richard Alan Peters II 58 −1 −1 −1
−1 8 −1
−1 −1 −1 Convolution Examples: H + V + D Diff. 20110929 19992011 by Richard Alan Peters II 59 Convolution Examples: Original Images 20110929 19992011 by Richard Alan Peters II 60 ...
View
Full
Document
This note was uploaded on 12/06/2011 for the course EECE 253 taught by Professor Alanpeters during the Summer '07 term at Vanderbilt.
 Summer '07
 AlanPeters
 Electrical Engineering, Image processing

Click to edit the document details