KavitaBala Source S Lazebnik Announcements Prelim next Thu Everything till Monday Where are we Imaging pixels features Scenes geometry material lighting Recognition people objects ID: 816134
Download The PPT/PDF document "Lecture 18: Cameras CS4670 / 5670: Compu..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Lecture 18: Cameras
CS4670 / 5670: Computer Vision
KavitaBala
Source: S. Lazebnik
Slide2Announcements
Prelim next ThuEverything till Monday
Slide3Where are we?
Imaging: pixels, features, …Scenes: geometry, material, lightingRecognition: people, objects, …
Slide4Reading
Szeliski 2.1.3-2.1.6
Slide5Panoramas
Now we know how to create panoramas!Given two images:Step 1: Detect featuresStep 2: Match featuresStep 3: Compute a homography using RANSACStep 4: Combine the images together (somehow)What if we have more than two images?
Slide6Can we use
homographies to create a 360 panorama?To figure this out, we need to learn what a camera is
Slide7360 panorama
Slide8Slide9Image formation
Let’s design a cameraIdea 1: put a piece of film in front of an objectDo we get a reasonable image?
Slide10Pinhole camera
Add a barrier to block off most of the raysThis reduces blurringThe opening known as the apertureHow does this transform the image?
Slide11Camera Obscura
Basic principle known to Mozi (470-390 BC), Aristotle (384-322 BC)Drawing aid for artists: described by Leonardo da Vinci (1452-1519)
Gemma Frisius, 1558Source: A. Efros
Slide12Camera Obscura
Slide13Pinhole photography
Justin Quinnell, The Clifton Suspension Bridge. December 17th 2007 - June 21st 20086-month exposure
Slide14Shrinking the aperture
Why not make the aperture as small as possible?
Less light gets through
Diffraction effects...
Slide15Shrinking the aperture
Slide16Adding a lens
A lens focuses light onto the filmThere is a specific distance at which objects are “in focus”other points project to a “circle of confusion” in the imageChanging the shape of the lens changes this distance
“circle of
confusion”
Slide17Lenses
A lens focuses parallel rays onto a single focal pointfocal point at a distance f beyond the plane of the lens (the focal length)f is a function of the shape and index of refraction of the lensAperture restricts the range of raysaperture may be on either side of the lensLenses are typically spherical (easier to produce)
focal point
F
Slide18The eye
The human eye is a cameraIris - colored annulus with radial musclesPupil - the hole (aperture) whose size is controlled by the irisWhat’s the “film”?
photoreceptor cells (rods and cones) in the retina
Slide19http://
www.telegraph.co.uk/news/earth/earthpicturegalleries/7598120/Animal-eyes-quiz-Can-you-work-out-which-creatures-these-are-from-their-eyes.html?image=25
Slide20http://
www.telegraph.co.uk/news/earth/earthpicturegalleries/7598120/Animal-eyes-quiz-Can-you-work-out-which-creatures-these-are-from-their-eyes.html?image=25
Slide21Eyes in nature:
eyespots to pinhole camera
http://upload.wikimedia.org/wikipedia/commons/6/6d/Mantis_shrimp.jpg
Slide22Projection
Slide23Projection
Slide24Müller-Lyer Illusion
http://www.michaelbach.de/ot/sze_muelue/index.html
Slide25Modeling projection
The coordinate systemWe will use the pinhole model as an approximationPut the optical center (Center Of Projection) at the originPut the image plane (Projection P
lane) in front of the COPWhy?The camera looks down the negative z axiswe need this if we want right-handed-coordinates
Slide26Modeling projection
Projection equationsCompute intersection with PP of ray from (x,y,z) to COPDerived using similar triangles (on board)
The screen-space or image-plane projection is therefore:
Slide27Modeling projection
Is this a linear transformation?homogeneous image coordinates
homogeneous scene coordinatesConverting from homogeneous coordinates
no—division by z is nonlinear
Slide28Perspective Projection
Projection is a matrix multiply using homogeneous coordinates:divide by third coordinate
This is known as
perspective projection
The matrix is the projection matrix
(Can also represent as a 4x4 matrix – OpenGL does something like this)
Slide29Perspective Projection
How does scaling the projection matrix change the transformation?
Slide30Orthographic projection
Special case of perspective projectionDistance from the COP to the PP is infiniteGood approximation for telephoto opticsAlso called “parallel projection”: (x, y, z) → (x, y)
What’s the projection matrix?
Image
World
Slide31Figures © Stephen E. Palmer, 2002
Dimensionality Reduction Machine (3D to 2D)
3D world2D image
What have we lost?AnglesDistances (lengths)
Slide by A. Efros
Slide32Projection properties
Many-to-one: any points along same ray map to same point in imagePoints → pointsLines → lines (collinearity is preserved)But line through focal point projects to a pointPlanes → planes (or half-planes)But plane through focal point projects to line
Slide33Projection properties
Parallel lines converge at a vanishing pointEach direction in space has its own vanishing pointBut parallels parallel to the image plane remain parallel
Slide34Orthographic projection
Slide35Perspective projection
Slide36Camera parameters
How many numbers do we need to describe a camera?We need to describe its pose in the worldWe need to describe its internal parameters
Slide37A Tale of Two Coordinate Systems
“The World”Camera
x
y
z
v
w
u
o
COP
Two important coordinate systems:
1.
World
coordinate system
2.
Camera
coordinate system
Slide38Camera parameters
To project a point (x,y,z) in world coordinates into a cameraFirst transform (x,y,z) into camera coordinates
Need to knowCamera position (in world coordinates)Camera orientation (in world coordinates)Then project into the image planeNeed to know camera intrinsicsThese can all be described with matrices
Slide39Projection equation
The projection matrix models the cumulative effect of all parameters
Camera parameters
Slide40Projection equation
The projection matrix models the cumulative effect of all parameters
Camera parameters
A camera is described by several parameters
Translation
T
of the optical center from the origin of world
coords
Rotation
R
of the image plane
focal length
f
,
principal
point
(
x’
c
,
y’
c
)
, pixel size
(
s
x
,
s
y
)
blue parameters are called “
extrinsics
,” red are “
intrinsics
”
Slide41Projection equation
The projection matrix models the cumulative effect of all parametersUseful to decompose into a series of operations
projection
intrinsics
rotation
translation
identity matrix
Camera parameters
A camera is described by several parameters
Translation
T
of the optical center from the origin of world
coords
Rotation
R
of the image plane
focal length
f
,
principal
point
(
x’
c
,
y’
c
)
, pixel size
(
s
x
,
s
y
)
blue parameters are called “
extrinsics
,” red are “
intrinsics
”
The definitions of these parameters are
not
completely standardized
especially intrinsics—varies from one book to another
Slide42Projection matrix
0
(in homogeneous image coordinates)
Slide43Extrinsics
How do we get the camera to “canonical form”?(Center of projection at the origin, x-axis points right, y-axis points up, z-axis points backwards)
0
Slide44Affine change of coordinates
Coordinate frame: point plus basisNeed to change representation ofpoint from one basis to anotherCanonical: origin (0,0) w/ axes e1, e2
Recap from 4620
0
Slide45Another way of thinking about this
Change of coordinates
Recap from 46200
Slide46Coordinate frame summary
Frame = point plus basisFrame matrix (frame-to-canonical) isMove points to and from frame by multiplying with F
Recap from 4620
0
Slide47Extrinsics
How do we get the camera to “canonical form”?(Center of projection at the origin, x-axis points right, y-axis points up, z-axis points backwards)
0
Step 1: Translate by -
c
Slide48Extrinsics
How do we get the camera to “canonical form”?(Center of projection at the origin, x-axis points right, y-axis points up, z-axis points backwards)
0
Step 1: Translate by -
c
Slide49Extrinsics
How do we get the camera to “canonical form”?(Center of projection at the origin, x-axis points right, y-axis points up, z-axis points backwards)
0
Step 1: Translate by -
c
Step 2: Rotate by
R
3x3
rotation matrix
Slide50Extrinsics
How do we get the camera to “canonical form”?(Center of projection at the origin, x-axis points right, y-axis points up, z-axis points backwards)
0
Step 1: Translate by -
c
Step 2: Rotate by
R
Slide51Perspective projection
(intrinsics)
(converts from 3D rays in camera coordinate system to pixel coordinates)
Slide52Perspective projection
(intrinsics)
in general,
:
aspect ratio
(1 unless pixels are not square)
:
skew
(0 unless pixels are shaped like rhombi/parallelograms)
:
principal point
((0,0) unless optical axis doesn’t intersect projection plane at origin)
(upper triangular matrix)
(converts from 3D rays in camera coordinate system to pixel coordinates)
Slide53Projection matrix
(
t
in book’s notation)
translation
rotation
projection
intrinsics