CSE 473/573 - PowerPoint Presentation

Slide1

CSE 473/573 Computer Vision and Image Processing (CVIP)

Ifeoma

Nwogu

inwogu@buffalo.edu

Lecture 4 – Image formation(part I)Slide2

ScheduleLast class linear algebra

overview

Today

Image formation and camera properties

Readings for today: Forsyth and Ponce 1.1,

1.4,

Szeliski

2.1 and 2.3.1 (optional).Slide3

Physical parameters of image formation

Optical

Sensor’s lens type

focal length, field of view, aperture

Geometric

Type of projection

Camera pose

Photometric

Type, direction, intensity of light reaching sensor

Surfaces’ reflectance propertiesSlide4

What is an image?Till now: a

function –

a 2D pattern of intensity values

Today

: a 2D projection of 3D

points

What is a camera?

Some device that allows the

projection

of light from 3D points to some “medium” that will record the light pattern. Slide5

1st known photograph

Heliograph-

a pewter plate coated with bitumen of Judea (an asphalt derivative of petroleum); after at least a day-long exposure of eight hours, the plate was removed and the latent image of the view from the window was rendered visible by washing it with a mixture of oil of lavender and white petroleum which dissolved away the parts of the bitumen which had not been hardened by light. – Harry Ransom Center UT Austin

View from the Window at le Gras,

Joseph

Nicéphore

Niépce

1826 Reproduction, 1952 Slide6

Image formation

Let’s design a camera:

Put a film in front of an object

Will we get a reasonable image?

Why? Why not?Slide7

Turning a room into a camera obscura

A.

Torralba

and W. Freeman,

Accidental Pinhole and

Pinspeck

Cameras

, CVPR 2012

Hotel room, contrast enhancedView from hotel windowAccidental pinholes produce images that are unnoticed or misinterpreted as shadowsSlide8

Image formation

Let’s design a camera:

Put a film in front of an object

Add a barrier with an opening to block off most of the rays (reduce blurring)

Opening is called

apertureSlide9

Ist known camera

Known to Aristotle (384-322 B.C.)

According to

DaVinci

“When images of illuminated objects ... penetrate through a small hole into a very dark room ... you will see [on the opposite wall] these objects in their proper form and color, reduced in size, in a reversed position, owing to the intersection of the rays".

Depth of the room is the “focal length”

How does the aperture size affect the image?Slide10

Shrinking the aperture

Slide by Steve Seitz

Pinhole too big -

many directions are

averaged, blurring the

image

Pinhole too small-

diffraction effects blur

the image

Generally, pinhole

cameras are

dark

, because

a very small set of rays

from a particular point

hits the screen.Slide11

Shrinking the aperture

Pinhole too big -

many directions are

averaged, blurring the

image

Pinhole too small-

diffraction effects blur

the image

Generally, pinhole

cameras are

dark

, because

a very small set of rays

from a particular point

hits the screen.Slide12

A

lens focuses light onto

the

film

There

is a

specific distance

at which objects are “in focus”

other

points project to a “circle of confusion” in the image

Changing

the shape or relative locations of the lens elements changes this distance

Adding a lens - concept of focusSlide13

The thin lensSlide14

The thin lens

Sign is +

ve

when incident lens surface is convex, and –

ve

when concaveSlide15

Depth of field

http://www.cambridgeincolour.com/tutorials/depth-of-field.htm

Slide by A. Efros

Depth of field is the range of distance within the subject that is acceptably sharp.Slide16

How can we control the depth of field?

Changing the aperture size affects depth of field

A smaller aperture increases the range in which the object is approximately in focus

But small aperture reduces amount of light – need to increase exposure

Slide by A. EfrosSlide17

Field of View (FOV)FOV is the extent of the observable world that is seen at any given moment.

For cameras, it is a solid angle through which a detector is sensitive to

light

the

area of the inspection

captured

on the camera’s imager.Slide18

Zooming and Moving are not the same…

Large

FOV, small f

Camera close to car

Small

FOV, large f

Camera far from the car Slide19

Real lens systemsSlide20

Lens flaws: chromatic aberration

A lens can have different refractive indices for different wavelengths: causes color fringing

Near Lens Center

Near Lens Outer EdgeSlide21

Lens flaws: Spherical aberration

Spherical lenses don

’

t focus light perfectly

Rays farther from the optical axis focus closerSlide22

Lens flaws: Spherical aberration

Left:

image showing low level of spherical aberration and

right:

image showing high level of spherical aberration

http://www.mto-ophtalmo.ch/intraocular-lenses/neutral-asphericity/Slide23

No distortion

Pin cushion

Barrel

Radial distortion

Caused by imperfect lenses

Deviations are most noticeable near the edge of the lensSlide24

Lens flaws: VignettingSlide25

Digital camera

A digital camera replaces film with a sensor array

Each cell in the array is light-sensitive diode that converts photons to electrons

Two common types

Charge Coupled Device

(CCD)

Complementary metal oxide semiconductor

(CMOS)http://electronics.howstuffworks.com/digital-camera.htm

Slide by Steve SeitzSlide26

CCD vs. CMOS

CCD:

transports the charge across the chip and reads it at one corner of the array. An

analog-to-digital converter (ADC)

then turns each pixel's value into a digital value by measuring the amount of charge at each photosite and converting that measurement to binary form

CMOS:

uses several transistors at each pixel to amplify and move the charge using more traditional wires. The CMOS signal is digital, so it needs no ADC.

http://www.dalsa.com/shared/content/pdfs/CCD_vs_CMOS_Litwiller_2005.pdf

http://electronics.howstuffworks.com/digital-camera.htmSlide27

Geometric projectionsSlide28

Types of 3D projections3D projection

is any method of mapping three-dimensional points to a two-dimensional

plane.

Perspective projections

objects in the distance appear smaller than those close

by

Parallel lines converge at an image point in infinity, on the horizon

Weak perspective projections

perspective effects, not over the scale of individual objectsOrthographic projectionsobjects in the distance appear same size as those close byparallel lengths at all points are of the same scale regardless of distance from the cameraSlide29

Distant objects are smaller

Effects of

perspective projection

:

Apparent size of object depends on their distance e.g. B’ and C’ have the same height but in reality A and C are half the size of B

Distance d from pinhole O to the plane of C is half the distance from O to plane of A and B.Slide30

Parallel lines meet

Projection of 2 parallel lines lying in the same plane:

The projections of 2 parallel lines in the same plane

F

appear to converge on h

h is a horizontal line formed by the intersection of image plane

P

and a plane parallel to

F passing through the aperture O.The line L in plane F and parallel to image plane P has no image

It is common

to draw

the image plane (or film)

in front

of the focal

point. Moving

the film plane

merely scales

the image.Slide31

Vanishing points

Each set of parallel lines (=direction) meets at a different point

The vanishing point for this direction

Sets of parallel lines on the same plane lead to collinear vanishing points.

The line is called the horizon for that plane

Good ways to spot faked images

scale and perspective don’t work

vanishing points behave badly

supermarket tabloids are a great source.Slide32

Example of a scene vanishing pointSlide33

Perspective projection

Consider a coordinate system (

O,

i

, j, k

) attached to the camera whose origin

O

coincides with the camera aperture.

O is located at a distance d along the vector k.The line passing through the aperture and perpendicular to P is the optical axisThe point c where this line intersects with the plane P is the image center. This is often the origin of the image plane coordinate frame.Slide34

Perspective projection equations

In image space,

z

=

d

Since

P

,

O, and p are collinear, Op = lOP for some l,x =

l

X

,

y

=

l

Y

,

d

=

l

Z

OR

l

=

=

Therefore,

x

=

d

and

y

=

d

 Slide35

Weak perspective

An even coarser approximation of image formation

Consider front-parallel plane

P

o

defined by

Z =

ZoFor any point P in Po x = -

m

X

,

y

=

-

m

Y

,

where

m

=

-

m

is the positive

magnification

associated with plane

P

o

 Slide36

Weak perspective

Issue

perspective effects, but not over the scale of individual objects

collect points into a group at about the same depth, then divide each point by the depth of its group

Advantage: easy

Disadvantage: wrongSlide37

Orthographic projection

No reversal of image features

m

= -1 (unnatural negative magnification)

All light rays are parallel to the

k

-axis and orthogonal to

P

x

=

X

,

y

=

Y

Useful for

creating to-scale

drawings for construction and

engineering (showing details)Slide38

Modeling projection

Projection equation:

Source: J. Ponce, S. Seitz

x

y

z

dSlide39

Homogeneous coordinatesIs this a linear transformation?

Trick: add one more coordinate:

homogeneous image

coordinates

homogeneous scene

coordinates

Converting

from

homogeneous coordinates

no—division by z is nonlinear

Slide by Steve SeitzSlide40

divide by the third coordinate

Perspective Projection Matrix

Projection is a matrix multiplication using homogeneous coordinates

In practice: lots of coordinate transformations…

World to

camera coord.

trans. matrix

(4x4)

Perspective

projection matrix

(3x4)

Camera to

pixel coord.

trans. matrix

(3x3)

=

2D

point

(3x1)

3D

point

(4x1)Slide41

Orthographic projection (sort of…)

http://glasnost.itcarlow.ie/~powerk/GeneralGraphicsNotes/projection/orthographicprojection.html

M.C. Escher's

waterfall Slide42

Orthographic ProjectionSpecial case of perspective projectionDistance from center of projection to image plane is infinite

Also called “parallel projection

”

What’s the projection matrix?

Slide by Steve SeitzSlide43

Physical parameters of image formation

Optical

Sensor’s lens type

focal length, field of view, aperture

Geometric

Type of projection

Camera pose

Photometric

Type, direction, intensity of light reaching sensorSurfaces’ reflectance propertiesSlide44

Slide CreditsDavid Forsyth – UIUC, slides accompanying Forsyth and Ponce – Computer Vision book, 2/e

Rob Fergus – NYU

AaronBobick

– GA Tech

Svetlana

Lazebnik

- UIUCSlide45

Next classMore on image f

ormation (photometric)

Readings for next lecture:

Forsyth and Ponce

2.1

,

2.2.4

;

Szeliski 2.2 (optional)Readings for today: Forsyth and Ponce 1.1, 1.4; Szeliski 2.1 and 2.3.1, (optional)Slide46

Questions