Chapter 14 The pinhole camera Structure Pinhole camera model Three geometric problems Homogeneous coordinates Solving the problems Exterior orientation problem Camera calibration 3D reconstruction ID: 366342
Download Presentation The PPT/PDF document "Computer vision: models, learning and in..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Computer vision: models, learning and inference
Chapter 14
The pinhole cameraSlide2
Structure
Pinhole camera model
Three geometric problems
Homogeneous coordinatesSolving the problemsExterior orientation problemCamera calibration3D reconstructionApplications
2
2
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide3
Motivation
3
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Sparse stereo reconstruction
Compute the depth at a set of sparse matching pointsSlide4
Pinhole camera
4
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Real camera image
is inverted
Instead model impossible but more convenient virtual imageSlide5
Pinhole camera terminology
5
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide6
Normalized Camera
6
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
By similar triangles:Slide7
Focal length parameters
7
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide8
Can model both
the effect of the distance to the focal plane
the density of the receptors
with a single focal length parameter f
In practice, the receptors may not be square:
So use different focal length parameter for x and y dims Focal length parameters8Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide9
Offset parameters
9
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Current model assumes that pixel (0,0) is where the principal ray strikes the image plane (i.e. the center)
Model offset to center Slide10
Finally, add skew parameter
Accounts for image plane being not exactly perpendicular to the principal ray
Skew parameter
10
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide11
Position
w
=(
u,v,w)T
of point in the world is generally not expressed in the frame of reference of the camera.Transform using 3D transformation
or Position and orientation of camera11Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Point in frame of reference of camera
Point in frame of reference of worldSlide12
Intrinsic parameters (stored as intrinsic matrix)
Extrinsic parameters
Complete pinhole camera model
12
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide13
For short:
Add noise – uncertainty in localizing feature in image
Complete pinhole camera model
13
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide14
Radial distortion
14
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide15
Structure
Pinhole camera model
Three geometric problems
Homogeneous coordinatesSolving the problemsExterior orientation problemCamera calibration3D reconstructionApplications
15
15
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide16
Problem 1: Learning extrinsic parameters (exterior orientation)
16
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Use maximum likelihood:Slide17
Problem 2 – Learning intrinsic parameters (calibration)
17
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Use maximum likelihood:Slide18
Calibration
18
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Use 3D target with known 3D points Slide19
Problem 3 – Inferring 3D points (triangulation / reconstruction)
19
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Use maximum likelihood:
TO ChangeSlide20
Solving the problems
20
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
None of these problems can be solved in closed form
Can apply non-linear optimization to find best solution but slow and prone to local minima
Solution – convert to a new representation (homogeneous coordinates) where we can solve in closed form.Caution! We are not solving the true problem – finding global minimum of wrong* problem. But can use as starting point for non-linear optimization of true problem*= We’ll first minimize algebraic error, instead of geometric error (see Minimizing Algebraic Error in Geometric Estimation Problems, Hartley 1998)Slide21
Structure
Pinhole camera model
Three geometric problems
Homogeneous coordinatesSolving the problemsExterior orientation problemCamera calibration3D reconstructionApplications
21
21
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide22
Homogeneous coordinates
22
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Convert 2D coordinate to 3D
To convert backSlide23
Geometric interpretation of homogeneous coordinates
23
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide24
Pinhole camera in
homogeneous coordinates
24
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Camera model:
In homogeneous coordinates:
(linear!)Slide25
Pinhole camera in
homogeneous coordinates
25
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Writing out these three equations
Eliminate l to retrieve original equationsSlide26
Adding in extrinsic parameters
26
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Or for short:
Or even shorter:Slide27
Structure
Pinhole camera model
Three geometric problems
Homogeneous coordinatesSolving the problemsExterior orientation problemCamera calibration3D reconstructionApplications
27
27
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide28
Problem 1: Learning extrinsic parameters (exterior orientation)
28
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Use maximum likelihood:Slide29
Exterior orientation
29
29
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Start with camera equation in homogeneous coordinates
Pre-multiply both sides by inverse of camera calibration matrixSlide30
Exterior orientation
30
30
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
The third equation gives us an expression for
l
Substitute back into first two linesSlide31
Exterior orientation
31
31
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Linear equation – two equations per point – form system of equationsSlide32
Exterior orientation
32
32
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Minimum direction problem of the form , Find minimum of subject to .
To solve, compute the SVD and then set to the last column of . Slide33
Exterior orientation
33
33
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Now we extract the values of and from .
Problem: the scale is arbitrary and the rows and columns of the rotation matrix may not be orthogonal.
Solution: compute SVD and then choose .
Use the ratio between the rotation matrix before and after to rescale
Use these estimates for start of non-linear optimisation.Slide34
Structure
Pinhole camera model
Three geometric problems
Homogeneous coordinatesSolving the problemsExterior orientation problemCamera calibration3D reconstructionApplications
34
34
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide35
Problem 2 – Learning intrinsic parameters (calibration)
35
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Use maximum likelihood:Slide36
One approach (not very efficient) is to alternately
Optimize extrinsic parameters for fixed intrinsic
Optimize intrinsic parameters for fixed extrinsic
Calibration
36
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince( Then use non-linear optimization )Slide37
Intrinsic parameters
37
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Maximum likelihood approach
This is a least squares problem.Slide38
Intrinsic parameters
38
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
The function is linear
w.r.t
. intrinsic parameters. Can be written in form
Now solve least squares problemSlide39
Structure
Pinhole camera model
Three geometric problems
Homogeneous coordinatesSolving the problemsExterior orientation problemCamera calibration3D reconstructionApplications
39
39
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide40
Problem 3 – Inferring 3D points (triangulation / reconstruction)
40
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Use maximum likelihood:Slide41
Reconstruction
41
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Write j
th pinhole camera in homogeneous coordinates:
Pre-multiply with inverse of intrinsic matrixSlide42
Reconstruction
42
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Last equations gives
Substitute back into first two equations
Re-arranging get two linear equations for
[
u,v,w
]
Solve using >1 cameras and then use non-linear optimizationSlide43
Structure
Pinhole camera model
Three geometric problems
Homogeneous coordinatesSolving the problemsExterior orientation problemCamera calibration3D reconstructionApplications
43
43
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide44
Depth from structured light
44
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide45
Depth from structured light
45
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide46
Depth from structured light
46
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide47
Shape from silhouette
47
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide48
Shape from silhouette
48
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide49
Shape from silhouette
49
Computer vision: models, learning and inference. ©2011 Simon J.D. PrinceSlide50
Conclusion
50
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Pinhole camera model is a non-linear function that takes points in 3D world and finds where they map to in image
Parameterized by intrinsic and extrinsic matricesDifficult to estimate intrinsic/extrinsic/depth because non-linear
Use homogeneous coordinates where we can get closed form solutions (initial sol’ns only)