of Review the paper
"Creating Architectural Models from Images"
Authors:David Liebowitz, Antonio Criminisi and Andrew Zisserman (EUROGRAPHICS '99 )
Amol Ambardekar
For the course in Advanced Computer Vision Under the guidance of Dr. Mircea Nicolescu
The problem and the strategy
It is not always possible to make measurements of a scene to allow reconstruction This is the case in some of the buildings where buildings are destroyed or only archive images are available Authors have developed the methods to create 3D graphical models of scenes from a limited number of images These methods use geometric relationships that are common in architectural scenes such as parallelism and orthogonality By using circular points of a plane, simple linear algorithms are given to do the task
Methods discussed in the paper
Metric rectification of individual planes Computation of relative perpendicular distances from partially rectified planes Camera calibration Piecewise reconstruction of the planar object from a single view Metric rectification of 3D reconstructions from two views
Plane Rectification and Homography
Plane rectification is equivalent to obtaining an image of the world plane where camera's image plane and world plane is parallel A map between a world plane and a perspective image is a "Homography" H Points on image plane, x, are mapped to the points on the world plane x', as x' = Hx
Plane Rectification
Images from : http://www.robots.ox.ac.uk/~dl/planar.html
Metric rectification of a plane I
A homography is used for a plane rectification and can be decomposed into two transformations: H = MN M is the metric part of the homography and N is the nonmetric part There are two ways to rectify a plane:
The metric information is retained for both rectification methods The second method (used in the paper) does not determine the metric component M
Determine H using four planar points from the scene for which we know their world coordinates and their corresponding positions in the image Determine N by projecting the circular points into the image or equivalently using Stratified metric rectification
Metric Rectification of a plane II
N is the nonmetric part and is represented as follows:
Plane rectification is thus reduced to a four parameter problem since we require only N to metric rectify an image N can be further decomposed in two matrices N = AP, where
N is determined using stratified metric rectification method
Metric Rectification of a plane III
Stratified metric rectification has two parts
Projective to Affine Affine to Metric
P is determined in the first part by identifying the vanishing line l = (l1, l2, 1)
l
u
Metric Rectification of a plane IV
A is determined in the second part Rectangles or two orthogonal directions provide a pair of vanishing points in orthogonal directions This constraint is only enough to restore angles To determine length ratios for nonparallel lines, we need to determine A11= = 1/ We need one more constraint for the same, which can be obtained by
Finding a ratio of lengths in the orthogonal directions of the plane Determining natural camera parameters for which the principal point is known approximately
Results of metric rectification
Figure 2: The affine rectified image. Notice that parallelism is restored, but angles and ratios of lengths are still incorrect Figure 1: Original image
Figure 3: An incorrectly scaled image due to the aspect ratio ambiguity
Figure 4: The correctly scaled image, from the known length ratio of the sides of a window
Measuring distances of points from planes I
Given:
The vanishing line of a reference plane The vanishing point for directions orthogonal to the plane A reference distance orthogonal to the plane
The paper gives an algorithm to calculate the orthogonal distance of any point from the reference plane
Measuring distances of points from planes II
We are trying to find out D d(a,b) is the distance between point a and point b
D = Dr d(v, b)d(t, b) - d(r, b)d(t, b) d(v, b)d(r, b) - d(r, b)d(t, b) v
t b
r D
u
Measuring distances of points from planes III
The technique is used to reconstruct 3D model from an old painting "La Flagellazione di Cristo" by Piero della Francesca (1416 1492). The people are represented by flat silhouettes since it is not possible to recover volume from one image.
Original Image
View of the model with the roof removed to show the relative positions of people and columns in the scene
Camera calibration
Camera calibration means determining internal parameters of the camera The camera calibration matrix K is generally represented as
An image point x is related to a point in the camera's coordinate system xc as x = Kxc The simplified or natural camera calibration matrix is
The image of the absolute conic
The image of the absolute conic is a conic defined as where, K is camera calibration matrix and is 3x3 symmetric matrix with five degrees of freedom A point x lying on the conic satisfies The calibration matrix K may be computed from by Cholesky decomposition The imaged circular points of any plane lie on thus each rectified plane provides two points on and so provides two of the five constraints necessary to determine determines orthogonality of rays back projected from image points. A pair of vanishing points u and v arising orthogonal from directions in the world satisfy
Camera calibration from orthogonal vanishing points
A typical real world application such as an image of a building provides three orthogonal directions, and thus three constraints on Therefore there are insufficient constraints to solve for the full five parameter model But, they are enough to determine the three parameter natural camera model The extra camera constraints of zero skew and unit aspect ratio provide additional constraints on K follows from by Cholesky decomposition
Camera calibration from rectified planes
If the four rectification parameters for a scene plane are computed, then the imaged circular points for that plane are known These points lie on , satisfying The real and imaginary part of I give two constraints Each rectified plane thus give two constraints With three rectified planes there are six points on and camera internal parameters are over constrained World planes used for rectification need not be orthogonal
Rectification of plane with known internal parameters
The vanishing point of the world plane intersects in the circular points corresponding to that plane Therefore, if K is known, any plane for which the vanishing line is known can be rectified The rectification ambiguity (aspect ratio) can be solved if the camera calibration is known The vanishing line and camera also determine the orientation of the world plane relative to the camera in the camera centered co ordinate xc as The relative orientations of two planes may be computed from their vanishing lines as
Rectification of plane with partial internal parameters
It is common to have aspect ratio ambiguity for plane rectification If natural camera is assumed with principal point at the image center then f (focal length) is the only parameter to be determined It can be determined using single constraint like the single orthogonal vanishing point pair
Original Image
Rectification where the relative scaling of vertical and horizontal directions assumes a natural camera with the principal point at the centre of the image
Single view reconstruction I
It is used to build 3D models which require the rectification of 2 or more planes The plane rectification technique discussed in the paper is ideal for the task of reconstructing models of buildings that have planar surfaces It is often not possible to reconstruct any planes until the camera calibration has been estimated The vanishing point method can be used to estimate natural camera parameters
Single view reconstruction II
Having computed the camera, the relative orientation of planes in the scene that are not orthogonal can be computed if their vanishing lines can be found Their relative positions and dimensions can be determined if the intersection of a pair of planes is visible in the image Relative size can be computed from the rectification of a distance between common points using the homographies of both planes
Single view reconstruction III
There are two ways to proceed with plane recification to reconstruct 3D model
Left facade is considered as a reference plane and rectified the first. The right facade and ground planes define 3D planes orthogonal to...