Computer Vision and Image Processing CVIP Ifeoma Nwogu inwogubuffaloedu Lecture 4 Image formationpart I Schedule Last class linear algebra overview Today Image formation and camera properties ID: 577352
Download Presentation The PPT/PDF document "CSE 473/573" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
CSE 473/573 Computer Vision and Image Processing (CVIP)
Ifeoma
Nwogu
inwogu@buffalo.edu
Lecture 4 – Image formation(part I)Slide2
ScheduleLast class linear algebra
overview
Today
Image formation and camera properties
Readings for today: Forsyth and Ponce 1.1,
1.4,
Szeliski
2.1 and 2.3.1 (optional).Slide3
Physical parameters of image formation
Optical
Sensor’s lens type
focal length, field of view, aperture
Geometric
Type of projection
Camera pose
Photometric
Type, direction, intensity of light reaching sensor
Surfaces’ reflectance propertiesSlide4
What is an image?Till now: a
function –
a 2D pattern of intensity values
Today
: a 2D projection of 3D
points
What is a camera?
Some device that allows the
projection
of light from 3D points to some “medium” that will record the light pattern. Slide5
1st known photograph
Heliograph-
a pewter plate coated with bitumen of Judea (an asphalt derivative of petroleum); after at least a day-long exposure of eight hours, the plate was removed and the latent image of the view from the window was rendered visible by washing it with a mixture of oil of lavender and white petroleum which dissolved away the parts of the bitumen which had not been hardened by light. – Harry Ransom Center UT Austin
View from the Window at le Gras,
Joseph
Nicéphore
Niépce
1826 Reproduction, 1952 Slide6
Image formation
Let’s design a camera:
Put a film in front of an object
Will we get a reasonable image?
Why? Why not?Slide7
Turning a room into a camera obscura
A.
Torralba
and W. Freeman,
Accidental Pinhole and
Pinspeck
Cameras
, CVPR 2012
Hotel room, contrast enhancedView from hotel windowAccidental pinholes produce images that are unnoticed or misinterpreted as shadowsSlide8
Image formation
Let’s design a camera:
Put a film in front of an object
Add a barrier with an opening to block off most of the rays (reduce blurring)
Opening is called
apertureSlide9
Ist known camera
Known to Aristotle (384-322 B.C.)
According to
DaVinci
“When images of illuminated objects ... penetrate through a small hole into a very dark room ... you will see [on the opposite wall] these objects in their proper form and color, reduced in size, in a reversed position, owing to the intersection of the rays".
Depth of the room is the “focal length”
How does the aperture size affect the image?Slide10
Shrinking the aperture
Slide by Steve Seitz
Pinhole too big -
many directions are
averaged, blurring the
image
Pinhole too small-
diffraction effects blur
the image
Generally, pinhole
cameras are
dark
, because
a very small set of rays
from a particular point
hits the screen.Slide11
Shrinking the aperture
Pinhole too big -
many directions are
averaged, blurring the
image
Pinhole too small-
diffraction effects blur
the image
Generally, pinhole
cameras are
dark
, because
a very small set of rays
from a particular point
hits the screen.Slide12
A
lens focuses light onto
the
film
There
is a
specific distance
at which objects are “in focus”
other
points project to a “circle of confusion” in the image
Changing
the shape or relative locations of the lens elements changes this distance
Adding a lens - concept of focusSlide13
The thin lensSlide14
The thin lens
Sign is +
ve
when incident lens surface is convex, and –
ve
when concaveSlide15
Depth of field
http://www.cambridgeincolour.com/tutorials/depth-of-field.htm
Slide by A. Efros
Depth of field is the range of distance within the subject that is acceptably sharp.Slide16
How can we control the depth of field?
Changing the aperture size affects depth of field
A smaller aperture increases the range in which the object is approximately in focus
But small aperture reduces amount of light – need to increase exposure
Slide by A. EfrosSlide17
Field of View (FOV)FOV is the extent of the observable world that is seen at any given moment.
For cameras, it is a solid angle through which a detector is sensitive to
light
the
area of the inspection
captured
on the camera’s imager.Slide18
Zooming and Moving are not the same…
Large
FOV, small f
Camera close to car
Small
FOV, large f
Camera far from the car Slide19
Real lens systemsSlide20
Lens flaws: chromatic aberration
A lens can have different refractive indices for different wavelengths: causes color fringing
Near Lens Center
Near Lens Outer EdgeSlide21
Lens flaws: Spherical aberration
Spherical lenses don
’
t focus light perfectly
Rays farther from the optical axis focus closerSlide22
Lens flaws: Spherical aberration
Left:
image showing low level of spherical aberration and
right:
image showing high level of spherical aberration
http://www.mto-ophtalmo.ch/intraocular-lenses/neutral-asphericity/Slide23
No distortion
Pin cushion
Barrel
Radial distortion
Caused by imperfect lenses
Deviations are most noticeable near the edge of the lensSlide24
Lens flaws: VignettingSlide25
Digital camera
A digital camera replaces film with a sensor array
Each cell in the array is light-sensitive diode that converts photons to electrons
Two common types
Charge Coupled Device
(CCD)
Complementary metal oxide semiconductor
(CMOS)http://electronics.howstuffworks.com/digital-camera.htm
Slide by Steve SeitzSlide26
CCD vs. CMOS
CCD:
transports the charge across the chip and reads it at one corner of the array. An
analog-to-digital converter (ADC)
then turns each pixel's value into a digital value by measuring the amount of charge at each photosite and converting that measurement to binary form
CMOS:
uses several transistors at each pixel to amplify and move the charge using more traditional wires. The CMOS signal is digital, so it needs no ADC.
http://www.dalsa.com/shared/content/pdfs/CCD_vs_CMOS_Litwiller_2005.pdf
http://electronics.howstuffworks.com/digital-camera.htmSlide27
Geometric projectionsSlide28
Types of 3D projections3D projection
is any method of mapping three-dimensional points to a two-dimensional
plane.
Perspective projections
objects in the distance appear smaller than those close
by
Parallel lines converge at an image point in infinity, on the horizon
Weak perspective projections
perspective effects, not over the scale of individual objectsOrthographic projectionsobjects in the distance appear same size as those close byparallel lengths at all points are of the same scale regardless of distance from the cameraSlide29
Distant objects are smaller
Effects of
perspective projection
:
Apparent size of object depends on their distance e.g. B’ and C’ have the same height but in reality A and C are half the size of B
Distance d from pinhole O to the plane of C is half the distance from O to plane of A and B.Slide30
Parallel lines meet
Projection of 2 parallel lines lying in the same plane:
The projections of 2 parallel lines in the same plane
F
appear to converge on h
h is a horizontal line formed by the intersection of image plane
P
and a plane parallel to
F passing through the aperture O.The line L in plane F and parallel to image plane P has no image
It is common
to draw
the image plane (or film)
in front
of the focal
point. Moving
the film plane
merely scales
the image.Slide31
Vanishing points
Each set of parallel lines (=direction) meets at a different point
The vanishing point for this direction
Sets of parallel lines on the same plane lead to collinear vanishing points.
The line is called the horizon for that plane
Good ways to spot faked images
scale and perspective don’t work
vanishing points behave badly
supermarket tabloids are a great source.Slide32
Example of a scene vanishing pointSlide33
Perspective projection
Consider a coordinate system (
O,
i
, j, k
) attached to the camera whose origin
O
coincides with the camera aperture.
O is located at a distance d along the vector k.The line passing through the aperture and perpendicular to P is the optical axisThe point c where this line intersects with the plane P is the image center. This is often the origin of the image plane coordinate frame.Slide34
Perspective projection equations
In image space,
z
=
d
Since
P
,
O, and p are collinear, Op = lOP for some l,x =
l
X
,
y
=
l
Y
,
d
=
l
Z
OR
l
=
=
Therefore,
x
=
d
and
y
=
d
Slide35
Weak perspective
An even coarser approximation of image formation
Consider front-parallel plane
P
o
defined by
Z =
ZoFor any point P in Po x = -
m
X
,
y
=
-
m
Y
,
where
m
=
-
m
is the positive
magnification
associated with plane
P
o
Slide36
Weak perspective
Issue
perspective effects, but not over the scale of individual objects
collect points into a group at about the same depth, then divide each point by the depth of its group
Advantage: easy
Disadvantage: wrongSlide37
Orthographic projection
No reversal of image features
m
= -1 (unnatural negative magnification)
All light rays are parallel to the
k
-axis and orthogonal to
P
x
=
X
,
y
=
Y
Useful for
creating to-scale
drawings for construction and
engineering (showing details)Slide38
Modeling projection
Projection equation:
Source: J. Ponce, S. Seitz
x
y
z
dSlide39
Homogeneous coordinatesIs this a linear transformation?
Trick: add one more coordinate:
homogeneous image
coordinates
homogeneous scene
coordinates
Converting
from
homogeneous coordinates
no—division by z is nonlinear
Slide by Steve SeitzSlide40
divide by the third coordinate
Perspective Projection Matrix
Projection is a matrix multiplication using homogeneous coordinates
In practice: lots of coordinate transformations…
World to
camera coord.
trans. matrix
(4x4)
Perspective
projection matrix
(3x4)
Camera to
pixel coord.
trans. matrix
(3x3)
=
2D
point
(3x1)
3D
point
(4x1)Slide41
Orthographic projection (sort of…)
http://glasnost.itcarlow.ie/~powerk/GeneralGraphicsNotes/projection/orthographicprojection.html
M.C. Escher's
waterfall Slide42
Orthographic ProjectionSpecial case of perspective projectionDistance from center of projection to image plane is infinite
Also called “parallel projection
”
What’s the projection matrix?
Slide by Steve SeitzSlide43
Physical parameters of image formation
Optical
Sensor’s lens type
focal length, field of view, aperture
Geometric
Type of projection
Camera pose
Photometric
Type, direction, intensity of light reaching sensorSurfaces’ reflectance propertiesSlide44
Slide CreditsDavid Forsyth – UIUC, slides accompanying Forsyth and Ponce – Computer Vision book, 2/e
Rob Fergus – NYU
AaronBobick
– GA Tech
Svetlana
Lazebnik
- UIUCSlide45
Next classMore on image f
ormation (photometric)
Readings for next lecture:
Forsyth and Ponce
2.1
,
2.2.4
;
Szeliski 2.2 (optional)Readings for today: Forsyth and Ponce 1.1, 1.4; Szeliski 2.1 and 2.3.1, (optional)Slide46
Questions