791E CS Computer Vision
Instructor: Mircea Nicolescu Lecture 19
2D Geometric Transformations
General form of a transformation matrix Affine transformations
- Involve translations, rotations, scale, and shear - Preserve parallelism of lines but not lengths and angles
a11 a 21 a31
a12 a22 a32
a13 a23 a33
2
2D Geometric Transformations
Similarity transformations
- Involve rotation, translation, uniform scaling - Preserve angles and length ratios
Rigid transformations
- Involve only translations and rotations - Preserve areas, angles and lengths
3
2D Geometric Transformations
Rigid transformations cont.
- Property: the upper 2x2 submatrix is orthonormal
- Example:
4
2D Geometric Transformations
Shear transformations
- Change the shape of the object. - Shear along the x-axis: - Shear along the y-axis:
5
3D Geometric Transformations
Coordinate systems
- Right-handed vs. lefthanded systems
- Positive rotation angles for right-handed systems
6
3D Geometric Transformations
Homogeneous coordinates of a 3D point
- Idea: add a third coordinate: (x, y, z) (xh, yh, zh, w) - Homogenize (xh, yh, zh, w):
- In general: (x, y, z) (xw, yw, zw, w) (i.e., xh=xw, yh=yw, zh=zw) - - - - w can assume any value (w 0), for example, w = 1: (x, y, z) (x, y, z, 1) (no division required when you homogenize) (x, y, z) (2x, 2y, 2z, 2) (division required when you homogenize) Each point (x, y, z) corresponds to a line in the 4D-space of homogeneous coordinates
7
3D Geometric Transformations
Translation
8
3D Geometric Transformations
Scaling
9
3D Geometric Transformations
Rotation
- Rotation about the z-axis
10
3D Geometric Transformations
Rotation
- Rotation about the x-axis
11
3D Geometric Transformations
Rotation
- Rotation about the y-axis
12
3D Geometric Transformations
Change of coordinate systems
- Suppose that you know the coordinates of P3 in the xyz system and you need its coordinates in the RxRyRz system.
- You need to recover the transformation T from RxRyRz to xyz. - Apply T on P3 to compute its coordinates in the RxRyRz system.
13
3D Geometric Transformations
Change of coordinate systems cont.
- Assume that ux, uy, uz are the unit vectors in the xyz coordinate system. - Assume that rx, ry, rz are the unit vectors in the RxRyRz coordinate system (note: rx, ry, rz are represented in the xyz coordinate system). - Find a mapping rz uz, rx ux, and ry uy
14
3D Geometric Transformations
Change of coordinate systems cont.
- Verify:
15
Singular Value Decomposition (SVD)
Definition
- Any real mxn matrix A can be decomposed uniquely as:
A = UDVT
- U is mxn and column orthogonal (its columns are eigenvectors of AAT)
(AAT = UDVT VDUT = UD2UT )
- V is nxn and orthogonal (its columns are eigenvectors of ATA)
(ATA = VDUT UDVT = VD2VT )
- D is nxn diagonal (non-negative real values called singular values) D = diag(1, 2, ..., n) ordered so that 1 2 ... n (if is a singular value of A, its square is an eigenvalue of ATA)
16
Singular Value Decomposition (SVD)
Definition cont.
- If U = (u1 u2 . . . un) and V = (v1 v2 . . . vn), then
(actually, the sum goes from 1 to r where r is the rank of A)
Example:
- The eigenvalues of AAT, ATA are:
17
Singular Value Decomposition (SVD)
Example cont.
- The eigenvectors of AAT, ATA are:
- The expansion of A is:
- Important: note that the second eigenvalue is much smaller than the first; if we neglect it from the above summation, we can represent A by introducing relatively small errors:
18
Singular Value Decomposition (SVD)
Computing the rank using SVD
- The rank of a matrix = the number of non-zero singular values.
Computing the inverse of a matrix using SVD
- A square matrix A is nonsingular iff i 0 for all i - If A is a nxn nonsingular matrix, then its inverse is given by:
- If A is singular or ill-conditioned, then we can use SVD to approximate its inverse by the following matrix:
19
Singular Value Decomposition (SVD)
The condition of a matrix
- Consider the system of linear equations
- If small changes in b can lead to relatively large changes in the solution x, then A is ill-conditioned. - The ratio given below is related to the condition of A and measures the degree of singularity of A (the larger this value is, the closer A is to being singular)
20
Singular Value Decomposition (SVD)
Least squares solutions of mxn systems
- Consider the over-determined system of linear equations:
- Let r be the residual vector for some x:
- The vector x* which yields the smallest possible residual is called a least-squares solution (it is an approximate solution).
21
Singular Value Decomposition (SVD)
Least squares solutions of mxn systems cont.
- Although a least-squares solution always exist, it might not be unique - The least-squares solution x with the smallest norm ||x|| is unique and is given by:
Example
22
Singular Value Decomposition (SVD)
Computing A+ using SVD
- If ATA is ill-conditioned or singular, we can use SVD to obtain a least squares solution as follows:
x = ( AT A) -1 AT b VD0-1U T b
Least squares solutions of nxn systems
- If A is ill-conditioned or singular, SVD can give us a workable solution in this case too:
23
Singular Value Decomposition (SVD)
Homogeneous systems
- If b=0 then the linear system is called homogeneous:
- The minimum-norm solution in this case is x=0 (trivial solution). - For homogeneous linear systems, the meaning of a least-squares solution is modified by imposing the constraint:
- This is a "constrained" optimization problem:
24
Singular Value Decomposition (SVD)
Homogeneous systems cont.
- The minimum-norm solution for homogeneous systems is not always unique. - Special case: rank(A) = n 1 solution is x = avn (m n 1, n=0) (a is a constant)
(vn is the last column of V corresponds to the smallest ) - General case: rank(A) = n k (m n k, n-k+1=...n=0)
solution is x = a1vn-k+1 + a2vn-k+2 + ... + akvn (ai is a constant with a12 + a22 + ... + ak2 = 1)
25
Reference Frames
Five reference frames are typically used for general problems in 3D scene analysis
26
Reference Frames
Object Coordinate Frame
- This is a 3D coordinate system: xb, yb, zb - It is used to model ideal objects in both computer graphics and computer vision. - It is needed to inspect an object (e.g., to check if a particular hole is in proper position relative to other holes) - Object coordinates do not change regardless how the object is placed in the scene. - Notation: (Xb, Yb, Zb)T
27
Reference Frames
World Coordinate Frame
- This is a 3D coordinate system: xw, yw, zw - The scene consists of object models that have been placed (rotated and translated) into the scene, yielding coordinates in the world coordinate system. - It is needed to relate objects in 3D (e.g., the image sensor tells the robot where to pick up a bolt and in which hole to insert it). - Notation: (Xw, Yw, Zw)T
28
Reference Frames
Camera Coordinate Frame
- This is a 3D coordinate system (xc, yc, zc axes) - Its purpose is to represent objects with respect to the location of the camera. - Notation: (Xc, Yc, Zc)T
Image Plane Coordinate Frame (CCD plane)
- This is a 2D coordinate system (xf, yf axes) - Describes coordinates of 3D points projected on the image plane. - The projection of A is point a. - Notation: (x, y)T
29
Reference Frames
Pixel Coordinate Frame
- This is a 2D coordinate system (r, c axes) - Each pixel in this frame has integer coordinates. - Point A gets projected to image point (ar, ac) where ar and ac are integer row and column. - Notation: (xim, yim)T
30
Reference Frames
Transformations between frames
31
Projection
Pinhole camera and perspective projection
- This is the simplest imaging device which, however, captures accurately the geometry of perspective projection. - Rays of light enters the camera through an infinitesimally small aperture. - The intersection of the light rays with the image plane form the image of the object. - Such a mapping from three dimensions onto two dimensions is called perspective projection.
32
Projection
A simplified geometric arrangement
- In general, the world and camera coordinate systems are not aligned. - To simplify the derivation of the perspective projection equations, we will make the following assumptions:
- the center of projection coincides with the origin of the world. - the camera axis (optical axis) is aligned with the world's z-axis. - avoid image inversion by assuming that the image plane is in front of the center of projection.
33
Projection
Terminology
- The model consists of a plane (image plane) and a 3D point O (center of projection). - The distance f between the image plane and the center of projection O is the focal length (the distance between the lens and the CCD array). - The line through O and perpendicular to the image plane is the optical axis. - The intersection of the optical axis with the image place is called principal point or image center. (note: the principal is point not always the "actual" center of the image)
34
Projection
The equations of perspective projection
- Using the following similar triangles:
35
Projection
The equations of perspective projection cont.
- Using matrix notation:
- Verify the correctness of the above matrix (homogenize using w = Z):
36
Projection
Properties of perspective projection
- Many-to-one mapping
- The projection of a point is not unique (any point on the line OP has the same projection).
37
Projection
Properties of perspective projection cont.
- Scaling/Foreshortening
- The distance to an object is inversely proportional to its image size.
- When a line (or surface) is parallel to the image plane, the effect of perspective projection is scaling. - When an line (or surface) is not parallel to the image plane, we use the term foreshortening to describe the projective distortion (the dimension parallel to the optical axis is compressed relative to the frontal dimension).
38
Projection
Properties of perspective projection cont.
- Effect of focal length
- As f gets smaller, more points project onto the image plane (wide-angle camera). - As f gets larger, the field of view becomes smaller (more telescopic).
- Lines, distances, angles
- Lines in 3D project to lines in 2D. - Distances and angles are not preserved. - Parallel lines do not in general project to parallel lines (unless they are parallel to the image plane).
39
Projection
Properties of perspective projection cont.
- Vanishing point
- parallel lines in space project perspectively onto lines that on extension intersect at a single point in the image plane called vanishing point or point at infinity. - (alternative definition) the vanishing point of a line depends on the orientation of the line and not on the position of the line. - the vanishing point of any given line in space is located at the point in the image where a parallel line through the center of projection intersects the image plane.
40
Projection
Properties of perspective projection cont.
- Vanishing line
- the vanishing points of all the lines that lie on the same plane form the vanishing line. - also defined by the intersection of a parallel plane through the center of projection with the image plane.
41
Projection
Orthographic Projection
- It is the projection of a 3D object onto a plane by a set of parallel rays orthogonal to the image plane. - It is the limit of perspective projection as f (f / Z 1)
42
Projection
Orthographic Projection cont.
- Using matrix notation:
- Verify the correctness of the above matrix (homogenize using w=1):
Properties of orthographic projection
- Parallel lines project to parallel lines. - Size does not change with distance from the camera.
43
Projection
Weak Perspective Projection
- Perspective projection is a non-linear transformation. - We can approximate perspective by scaled orthographic projection (linear transformation) if:
44
Projection
Weak Perspective Projection cont.
- The term f / Z is a scale factor now (every point is scaled by the same factor). - Using matrix notation:
- Verify the correctness of the above matrix (homogenize using w =Z ):
45
Camera Parameters
Assumptions made so far:
- Equations derived so far are valid only when:
- all distances are measured in the camera's reference frame. - the image coordinates have their origin at the principal point.
In general, the world and pixel coordinate systems are related by a set of physical parameters such as:
- - - - the focal length of the lens the size of the pixels the position of the principal point the position and orientation of the camera
46
Camera Parameters
Camera parameters
- Two types of parameters need to be recovered in order for us to reconstruct the 3D structure of a scene from the pixel coordinates of its image points:
- Extrinsic camera parameters: the parameters that define the location and orientation of the camera reference frame with respect to a known world reference frame. - Intrinsic camera parameters: the parameters necessary to link the pixel coordinates of an image point with the corresponding coordinates in the camera reference frame.
47
Camera Parameters
Extrinsic camera parameters
- Identify uniquely the transformation between the unknown camera reference frame and the known world reference frame. - Typically, determining these parameters means:
- finding the translation vector between the relative positions of the origins of the two reference frames. - finding the rotation matrix that brings the corresponding axes of the two frames into alignment (i.e., onto each other)
48
Camera Parameters
Extrinsic camera parameters cont.
- Using the extrinsic camera parameters, we can find the relation between the coordinates of a point P in world (Pw) and camera (Pc) coordinates:
49
Camera Parameters
Intrinsic camera parameters
- These are the parameters that characterize the optical, geometric, and digital characteristics of the camera:
- the perspective projection (focal length f ). - the transformation between image plane coordinates and pixel coordinates. - the geometric distortion introduced by the optics.
- From Camera Coordinates to Image Plane Coordinates
- Apply perspective projection:
50
Camera Parameters
Intrinsic camera parameters cont.
- From Image Plane Coordinates to Pixel Coordinates
- where (ox, oy) are...