1995_Viola_thesis_registrationMI

No variant of correlation could match these images

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: and the Transformation Figure 5.21: The three central slices of the CT data used in the MR-CT experiments. 130 5.2. MEDICAL REGISTRATION EXPERIMENTS AI-TR 1548 Figure 5.22: An initial condition for MR-CT registration by maximization of mutual information displayed as a checkerboard composite of the three central slices. Figure 5.23: A nal con guration for MR-CT registration by maximization of mutual information. The three central slices of the MRI data are shown with the edges from the registered CT data overlaid. 131 Paul A. Viola CHAPTER 5. ALIGNMENT EXPERIMENTS 5.3 View Based Recognition Experiments In the previous vision experiments we used knowledge of the physics of imaging to show that the surface normal of an object should be predictive of the intensity observed in an image. Unfortunately, in many experimental situations no three dimensional model is available. In these situations it is frequently the case that the only available information about an object is a collection of images taken under a variety conditions. One approach for solving problems like this is to use a collection of images as the model. This is often called a view based" approach since the model is made up of a number of views of the model object. Given a novel image of some object, each model image is compared to it in turn. If some model image is close enough" to the novel image, the model and novel image are considered aligned or recognized. One can signi cantly reduce the number of model images required by adding an a ne transformation to the comparison process. The novel image is then compared to each model image under a set of a ne transformations. The most commonly used comparison metric is correlation. As we saw in Section 4.1.1, correlation makes the assumption that the model and the image are identical or possibly related by linear function. In general the set of images that can arise from a single object under varying illumination is very broad. Figure 5.24 shows two images of the same object in the same pose. These images are very di erent and are in fact anti-correlated: bright pixels in the left image correspond to dark pixels in the right image; dark pixels in the left image correspond to bright pixels in the right image. No variant of correlation could match these images together. We have presented techniques based on entropy that can match both correlated and anticorrelated signals. These techniques require only that there is some consistent relationship between model and image. Discouragingly, it is not di cult to nd two images of the same object for which there is no consistent relationship. Figure 5.25 shows a novel image which is aligned with the two model images. Figure 5.26 contains two scatter plots of the pixel values in the novel image versus the pixel values in the model images. Clearly, there is no simple consistent relationship displayed in either of these graphs. EMMA could not be used to match this novel image to either model image. 132 5.3. VIEW BASED RECOGNITION EXPERIMENTS AI-TR 1548 Figure 5.24: Car Model Images Figure 5.25: A novel image of the car model. 5.3.1 Photometric Stereo By itself each model image does not contain enough information to constrain the match between image and model. However, it is well known that taken together a collection of images can be used to determine the 3D shape of an object. As we've seen the 3D shape is su cient to constrain the match between image and model. When multiple images of an object are available a technique called photometric stereo can be used to estimate its 3D shape Horn, 1986. Photometric stereo works with images which are taken from the same location but under di erent illumination conditions. It is assumed that detailed information both about illumination and surface properties are available for each image. As a result a re ectance map can be computed for each image. The re ectance map determines the relationship between the normals of an object and the intensities in an image. The re ectance map together with the intensity of a pixel acts as a constraint on the normal vector visible from that pixel. The allowable normals usually lie along a closed curve on the unit circle. From a second image, and its associated re ectance map, another set of allowable normals can be computed. By intersecting these constraints, two images are su cient to determine the surface normal at each pixel. From the normals the shape can be obtained through integration. 133 Paul A. Viola CHAPTER 5. ALIGNMENT EXPERIMENTS 2 1 1 Novel Image Novel Image 2 0 -1 0 -1 -1 0 1 Model Image 1 2 -1 0 1 Model Image 2 2 Figure 5.26: The relationship between pixels in the novel image and each of the model images. Once the shape of the object is determined, the correct alignment could be computed using the three dimensional version of EMMA. The imaging function of this new two stage process is: I T xi = F Gu1xi; r1; u2xi; r2; q where G is the photometric stereo function that takes two images and two re ectance maps and returns the shape, and F  is our origi...
View Full Document

Ask a homework question - tutors are online