Unformatted text preview: and the Transformation Figure 5.21: The three central slices of the CT data used in the MR-CT experiments. 130 5.2. MEDICAL REGISTRATION EXPERIMENTS AI-TR 1548 Figure 5.22: An initial condition for MR-CT registration by maximization of mutual information displayed as a checkerboard composite of the three central slices. Figure 5.23: A nal con guration for MR-CT registration by maximization of mutual information. The three central slices of the MRI data are shown with the edges from the registered
CT data overlaid. 131 Paul A. Viola CHAPTER 5. ALIGNMENT EXPERIMENTS 5.3 View Based Recognition Experiments In the previous vision experiments we used knowledge of the physics of imaging to show that
the surface normal of an object should be predictive of the intensity observed in an image.
Unfortunately, in many experimental situations no three dimensional model is available. In
these situations it is frequently the case that the only available information about an object
is a collection of images taken under a variety conditions. One approach for solving problems
like this is to use a collection of images as the model. This is often called a view based"
approach since the model is made up of a number of views of the model object. Given a novel
image of some object, each model image is compared to it in turn. If some model image is
close enough" to the novel image, the model and novel image are considered aligned or
recognized. One can signi cantly reduce the number of model images required by adding an
a ne transformation to the comparison process. The novel image is then compared to each
model image under a set of a ne transformations. The most commonly used comparison
metric is correlation. As we saw in Section 4.1.1, correlation makes the assumption that the
model and the image are identical or possibly related by linear function.
In general the set of images that can arise from a single object under varying illumination
is very broad. Figure 5.24 shows two images of the same object in the same pose. These
images are very di erent and are in fact anti-correlated: bright pixels in the left image
correspond to dark pixels in the right image; dark pixels in the left image correspond to
bright pixels in the right image. No variant of correlation could match these images together.
We have presented techniques based on entropy that can match both correlated and anticorrelated signals. These techniques require only that there is some consistent relationship
between model and image. Discouragingly, it is not di cult to nd two images of the same
object for which there is no consistent relationship. Figure 5.25 shows a novel image which
is aligned with the two model images. Figure 5.26 contains two scatter plots of the pixel
values in the novel image versus the pixel values in the model images. Clearly, there is no
simple consistent relationship displayed in either of these graphs. EMMA could not be used
to match this novel image to either model image.
132 5.3. VIEW BASED RECOGNITION EXPERIMENTS AI-TR 1548 Figure 5.24: Car Model Images Figure 5.25: A novel image of the car model. 5.3.1 Photometric Stereo
By itself each model image does not contain enough information to constrain the match
between image and model. However, it is well known that taken together a collection of
images can be used to determine the 3D shape of an object. As we've seen the 3D shape is
su cient to constrain the match between image and model.
When multiple images of an object are available a technique called photometric stereo can
be used to estimate its 3D shape Horn, 1986. Photometric stereo works with images which
are taken from the same location but under di erent illumination conditions. It is assumed
that detailed information both about illumination and surface properties are available for
each image. As a result a re ectance map can be computed for each image. The re ectance
map determines the relationship between the normals of an object and the intensities in an
The re ectance map together with the intensity of a pixel acts as a constraint on the
normal vector visible from that pixel. The allowable normals usually lie along a closed curve
on the unit circle. From a second image, and its associated re ectance map, another set
of allowable normals can be computed. By intersecting these constraints, two images are
su cient to determine the surface normal at each pixel. From the normals the shape can be
obtained through integration.
133 Paul A. Viola CHAPTER 5. ALIGNMENT EXPERIMENTS
2 1 1 Novel Image Novel Image 2 0 -1 0 -1 -1 0
Model Image 1 2 -1 0
Model Image 2 2 Figure 5.26: The relationship between pixels in the novel image and each of the model images.
Once the shape of the object is determined, the correct alignment could be computed
using the three dimensional version of EMMA. The imaging function of this new two stage
I T xi = F Gu1xi; r1; u2xi; r2; q
where G is the photometric stereo function that takes two images and two re ectance
maps and returns the shape, and F is our origi...
View Full Document
- Spring '10
- The Land, Probability distribution, Probability theory, probability density function, Mutual Information, Paul A. Viola