1 Photogrammetry
Lecture Introduction Mathematical Foundations
Overview of Photogrammetry Lectures
Four Lectures and one workshop session
Basic Maths of Photogrammetry Handling rotations, matrixes, finding the inner and outer orientation Relative and Absolute Orientations, Orthorectification, Block Adjustments, aero triangulation Softcopy Photogrammetry, Photogrammetry and GIS, Satellite Sensors.
Lecture Overview
What is Photogrammetry Historical Perspective Terminology Image Formation Central Projection Interior and Exterior Orientation Image Capture Practicalities
What is Photogrammetry?
Photogrammetry is the technology of obtaining reliable 3-D information about physical objects and the environment through processes of recording, measuring, and interpreting photographic images Two stereo Images allow reconstruction of the 3D shape
From Photographs we produce
Coordinates of real world points on objects Maps or plans
topographic maps
Orthorectified Imagery Digital Terrain Models (DTMs)
without contacting the surface to be measured, and at a predetermined accuracy. Accuracy is mainly determined by the scale of the photograph.
Photogrammetry is used where ever you need a 3-D reconstruction
map making; Geographic Information Systems (GIS); Surveying (Architecture, accident analysis) medical imaging (map body shape) vehicles (collision avoidance); mining and heavy industry (online Quality control) Virtual Reality (Simulators)
Historical Perspective
1859, Tournachon, uses balloon and photography for Napolean's reconnaissance in battle of Solferino. 1864, Aime Laussedat, the "Father of Photogrammetry" surveys Paris by rooftop photography. 1909, Eduard Dolezal, University of Vienna, founded International Society of Photogrammetry. 1920-1960 Analog reconstruction Instruments 1960-1990 Semiautomated (computerbased) systems 1995-: Digital Photogrammetry 1999 Continuous Coverage from Space Images
Historical Phases Of Photogrammetry
Analogue Instruments
reconstruct via optical and/or mechanical means the geometry of stereo image formation
Analytical Instruments
measure image coordinates on photographs using electro-optical or mechanical methods aided by human visual system Use mathematical models to compute 3D coordinates
Digital Instruments
convert images to digital format Use mathematical techniques to find matching points Use mathematical models to compute 3D coordinates Cost is lower because of lack of requirement for high accuracy electro-mechanical-optical linkages Accuracy
depends on the pixel size has not surpassed the accuracy of analytical methods
Matching still problematic in difficult areas
Central Projection
Fundamental Image Formation Maths O: Projection Centre (Camera Position) PP: Principal Point (intersection of optical axis and iamge) f: focal length, principal distance, camera constant P: point in space P: image of point P
negative
diapositive
If you knew
the exact location of the camera, O, the exact orientation of the image plane wrt to some coordinate system the distance PP-P(image coordinate)
The line upon which P falls is uniquely determined. But there is no way to tell where on the line that P falls. To determine Ps position exactly, an intersecting line is required. This is the basis of photogrammetry.
Principle of Stereo Photogrammetry
PP1 PP2 P2 O1 f O2
P1
P Position of P is uniquely determined by the intersection of lines P1O1 and P2O2
Central Projection Equations
z
By similar triangles xp = - f * xP /zP and yp = - f * yP /zP or xP /zP = - xp /f and yP /zP = - yp /f
y O f PP x p zP x P Image Plane
xP
P
Interior (Inner) Orientation
To make use of the central projection, we need a metric image and a good image coordinate system: we need to know
the location of PP, the principle point, in the image (with image coordinates x=0, y=0). The direction of the x axis the focal length f . These are usually found in the camera calibration process ( f and the image coordinates of fiducial marks are found and listed)
Example of a fiducial mark, In this case found in each corner of the photograph
Centre of Collimation
PP
The Principle Point is located in reference to the fiducial marks, which also define an image coordinate system
Metric Cameras
One needs special metric cameras with fiducial marks or other means to determine the camera parameters. (also need metrically stable camera and film); There are many varieties of fiducial mark employed by different cameras and manufacturers. They all provide a means of precisely identifying the location of the principle point, coordinate system (and possible camera/film deformations) Corner Fiducials Side Fiducials
Photo Coordinate System
A Photo coordinate system is defined in reference to the fiducial marks, and the position of the PP given in the defined coordinate system, as (x0,y0). Positive x is defined as the direction of flight
y y
(x0,y0)
x
x
Relating Photo Coordinates to Image Coordinates
Define
Photo Coordinates
origin is specified by the fiducial marks positive x in direction of flight
Image Coordinates
origin is at the Projection Centre positive x in direction of flight negative z in direction of photograph.
Relationship between Photo Coordinates (x,y) and Image Coordinates (x,y,z)
y y y y x
(x0,y0)
x
x
x = x - x0 x y = y - y0 z=-f
Exterior Orientation
It is insufficient simply to know how the image coordinates and photo coordinates are related. We must also know how the image coordinate system is related to a world coordinate system. This will require six parameters, a translation in X, Y and Z directions, and three rotational angles. The World Coordinate System is used to locate points in the physical world.
Geographical Coordinates: Latitude, Longitude, Elevation Map Grid: Easting, Northing, Height
Exterior Orientation
z y Projection Centre x Z (X0,Y0,Z0) y x
Y X
P
xterior Orientation
X0 - The X coordinate of the camera projection centre Y0 - The Y coordinate of the camera projection centre Z0 - The Z coordinate of the camera projection centre
The rotation of the image coordinate system around the X axis
The rotation of the image coordinate system around the Y axis
The rotation of the image coordinate system around the Z axis
Vertical Photography
Typically, in aerial photography, we try to make the rotation angles as close to zero as possible. The ideal case where all angles are zero is called vertical or normal photography, and these equations appl: XP = X0 + xP YP = Y0 + yP ZP = Z0 + zP Small deviations from the ideal must be accounted for in accurate work. When there are large deviations from vertical, the images are called oblique
Low Oblique images exclude the horizon High Oblique images include the horizon
Coordinate Systems
Photo Coordinates - origin at centre of collimation. Denoted by (x,y) in notes. Image Coordinate System - origin at centre of Projection. Denoted by
(xp,yp,-f) for the image point P (xp,yp ,zp) for the actual point P related by central projection
World Coordinate System - origin at specified location. Denoted by (X,Y,Z)
Terminology and Formulas
Photoscale: ps = h/f
h: Flying height above the ground f: The focal length
O f xI h I mage Plane y x
Image size on ground, S:
S = sps s: Image side in photograph
Ground Area, GA:
GA = S2 = s2 ps2
xP P
Example
Locate the approximate position of the fence junction given the following
h=245m,
Interior Orientation
f = 150mm, x0 = 0.03mm, y0 = -0.02mm.
Exterior Orientation
X0 = 60245m Y0 = 4025m Z0 = 270m ===0
Step One: Photo Coordinates
Measure the photo coordinates from the centre of collimation, obtaining
xp = 37mm, yp = 23mm
xp
yp
Step Two: Image Coordinates
Use the position of the principle y point given by the interior O orientation to convert to the image coordinate system. f x = x - x0 (x0,y0) xI P xp = 37 - 0.03 37mm y = y - y0 yp = 23 + 0.02 23mm z = -f xP zp = -150mm
z x x (xp,yp,zp) y
P (xp,yp,zp)
Step Three: Central Projection
xP = - zP xp /f (h/f ) xp (245m/150mm)37mm 60m yP = - zP yp /f (h/f ) yp (245m/150mm)23mm 38m
Since we dont know zp, use -h as an approximate value. This approximation is reasonable for flat terrain, but is not suitable for high accuracy work.
Step Four: World Coordinates
XP = X0 + xP
XP = 60245m + 60m = 60305m
YP = Y0 + yP
YP = 4025m + 38m = 4063m
ZP = Z 0 + zP
ZP = 270m + -245m = 25m
z Z (X0,Y0,Z0)
y x x P (xp,yp,) (xp,yp,zp) y
xI Y
xP X
P (xp,yp,zp)
(XP,YP,ZP) World Coordinates computed from Photo Coordinates
Summary of Procedure
Measure Photo coordinates Transform Photo coordinates to image coordinates using location of PP Transform image coordinates to coordinates of point using the central projection.
Since height of the point is unknown, must use -h as an approximation
Transform point to world coordinate system
Problems with this approach
h is only known approximately
Air-platform is continually height changing ground surface is constantly changing the photoscale changes with ground height. Photo is not equivalent to a map Photographs are never perfectly normal, the transformation to world coordinates should take this into account.
How are the interior and exterior orientations obtained? The Central Projection assumes a pin-hole camera. Real cameras have lenses which introduce distortion Photogrammetry has developed rigorous methods for dealing with all these problems.
Example - Effect of inaccurate h
Suppose that instead of 245m, as assumed in the last example, the true height above the ground at the fence corner is 220m.
What is the difference between the true position and the previous estimate?
f = 150mm f h2=220m
xp2 = xp h2 /f =37mm220m/150mm =54m xp Compare to xp1 = 60m
h1= 245m xp2 xp1 Error introduced into position calculation
The error in point location is therefore 6m in the x dimension, and can be expressed by ex = xp h1 /f - xp h2 /f = (h1 - h2) xp /f ey = (h1 - h2) yp /f Error in percentage terms: 100(h 1 - h2) / h1 = 100 25/245 = 10.2% Where h1 is the height value used and h2 is the actual height at the point
The need to know h
Obviously, without knowing the height of the point, accurate determination of its location is impossible. The solution to this dilemma is to use stereo photogrammetry, or two overlapping photographs containing the same point. As previously explained, this allows the exact height of the point to be computed. Then its exact position is easily computed as previously explained. This gives the further advantage of computing elevation as well as location for each point.
In the following example we still assume that we have obtained the correct interior and exterior orientations
Principle of Stereo Photogrammetry
PP1 PP2 P2 O1 f O2
P1
P Position of P is uniquely determined by the intersection of lines P1O1 and P2O2
Ideal Case
Consider the ideal case where photographs are separated by a distance B along the x axis. Both photographs are perfectly normal, the exterior orientations are known, and the location of the principle point is at (0,0) defn:The Stereo Baseline, B, is the distance between the projection centres of the two cameras in stereo photography.
z O2 x f zp = -h xp1 P1 xp2 P2 For Photo 1: xp = -zp xp1/f yp = -zp yp1/f For Photo 2: xp = B+ -zp xp2/f yp = -zp yp2/f (1) (2)
O1
B
(3) (4)
P (xp,zp)
(2) and (4) imply that yp1 = yp2
There is no y parallax All the height information is contained in the x parallax
Defn: Parallax is the difference between the image coordinates in each image of a point in a stereo pair. x Parallax is measured along the x dimension. y Parallax is measured along the y dimension. Parallax is also known as disparity. In previous example: x Parallax = xp1- xp2
Solve for zp using (1) and (3) Eliminate xp
-zp xp1/f = B+ -zp xp2/f -zp (xp1 - xp2 )/ f = B -zp = Bf / (xp1 - xp2) = Bf / p
(5)
Where p is the x parallax of the point P Once zp is found from (5), xp and yp are easily found from the other equations. E.G.
xp = -zp xp1/f yp = -zp yp2/f
Finding (xp ,yp ,zp ) for the point P gives its exact 3D location with respect to the projection centre of image 1. Knowing the location of camera 1 in world coordinates allows us to find the world coordinates of point P, as before
XP = X0 + xP YP = Y0 + yP ZP = Z0 + zP We are assuming for the moment that there is no rotation between the world coordinate system and the image coordinate system. In other words the exterior orientation parameters , and are all zero.
Z (X0,Z0) O1 f zp = -h
z
Stereo Restitution in World Coordinate System B P1 O2 x P2
xp1
xp2
P (xp,zp) X Xp = X0 + xp Zp = Z0 + zp
Although the Y dimension is not shown in the diagram, it is treated exactly the same as the X dimension. If the exterior orientation is not known, a correct 3D model can still be formulated, with respect to the projection centre of camera 1. If B is unknown, it doesnt matter, because it appears as a scale factor in all three coordinates.
The 3D model can be computed subject to this scale factor. Relative distances can be computed, but not absolute distances. Directions are identical between the relative model and the absolute model
Example - Stereo Restitution
Left Image Right Image
xp2 xp1 yp1 yp2
Exterior Orientation - Image 1 X0 = 60245m Y0 = 4025m Z0 = 270m ===0
Exterior Orientation - Image 2 X0 = 60297m Y0 = 4025m Z0 = 270m ===0
Calculate the elevation and the location of the fence junction in the World Coordinate System The Interior Orientation for each image will be the same, as we assume that the camera is the same, and that its interior orientation is stable.
f = 150mm, x0 = 0.03mm, y0 = -0.02mm.
Step One - Measure Photo Coordinates
Photo 1
xp1 = 37mm, yp1 = 23mm
Photo 2
xp2 = -3mm, yp2 = 21mm
Note that since yp1 yp2 , the case is not perfectly ideal. However, we will ignore this small difference for the purposes of illustration
Step Two :- Find Image Coordinates for each image z
x = x - x0 xp1 = 37 - 0.03 37mm xp2 = -3 - 0.03 -3mm y = y - y0 yp1 = 23 + 0.02 23mm yp2 = 21 + 0.02 21mm z = -f zp1 = zp2 = -150mm y O1 f (x0,y0) x x P1(xp1,yp1,zp1) y
P1 (xp,yp,zp)
z O2 P2
Step Three Calculate zp, elevation wrt Camera One
B x
First, find B the stereo Baseline from the exterior orientation of each image. B = X02 - X01 = 60297...