/
Linear Filters April  6 th Linear Filters April  6 th

Linear Filters April 6 th - PowerPoint Presentation

alida-meadow
alida-meadow . @alida-meadow
Follow
346 views
Uploaded On 2018-11-01

Linear Filters April 6 th - PPT Presentation

2017 Yong Jae Lee UC Davis Announcements PS0 out today due 414 Friday at 1159 pm Carefully read course website Signup for piazza 2 Plan for today Image formation Image noise Linear filters ID: 708821

credit slide kristen grauman slide credit grauman kristen filter gaussian filters seitz steve moving average image noise 890 490

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Linear Filters April 6 th" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Linear Filters

April 6th, 2017

Yong Jae Lee

UC DavisSlide2

AnnouncementsPS0 out today; due 4/14 Friday at 11:59 pmCarefully read course website

Sign-up for piazza2Slide3

Plan for todayImage formationImage noiseLinear filters

Examples: smoothing filtersConvolution / correlation3Slide4

Image Formation

4

Slide credit:

Derek

HoiemSlide5

Digital cameraA digital camera replaces film with a sensor array

Each cell in the array is light-sensitive diode that converts photons to electronshttp://electronics.howstuffworks.com/digital-camera.htm

5

Slide credit:

Steve SeitzSlide6

Digital imagesSample

the 2D space on a regular gridQuantize each sample (round to nearest integer)Image thus represented as a matrix of integer values.

2D

1D

6

Slide credit:

Kristen

Grauman

, Adapted from Steve SeitzSlide7

Digital images

7

Slide credit:

Derek

Hoiem

CMOS sensorSlide8

Digital color images

8

Slide credit:

Kristen

GraumanSlide9

Color Sensing: Bayer Grid

Estimate RGB at each cell from neighboring values

http://en.wikipedia.org/wiki/Bayer_filter

Slide by Steve SeitzSlide10

R

G

B

Color images, RGB color space

Digital color images

10

Slide credit:

Kristen

GraumanSlide11

Images in Matlab

Images represented as a matrixSuppose we have an NxM RGB image called “

im

im

(1,1,1) = top-left pixel value in R-channel

im

(y, x, b) = y pixels down, x pixels to right in the

b

th

channel

im

(N, M, 3) = bottom-right pixel in B-channel

imread

(filename) returns a uint8 image (values 0 to 255)

Convert to double format (values 0 to 1) with im2double

0.92

0.93

0.94

0.97

0.62

0.37

0.85

0.97

0.93

0.92

0.99

0.95

0.89

0.82

0.89

0.56

0.31

0.75

0.92

0.81

0.95

0.91

0.89

0.72

0.51

0.55

0.51

0.42

0.57

0.41

0.49

0.91

0.92

0.96

0.95

0.88

0.94

0.56

0.46

0.91

0.87

0.90

0.97

0.95

0.71

0.81

0.81

0.87

0.57

0.37

0.80

0.88

0.89

0.790.850.490.620.600.580.500.600.580.500.610.450.330.860.840.740.580.510.390.730.920.910.490.740.960.670.540.850.480.370.880.900.940.820.930.690.490.560.660.430.420.770.730.710.900.990.790.730.900.670.330.610.690.790.730.930.970.910.940.890.490.410.780.780.770.890.990.93

0.920.930.940.970.620.370.850.970.930.920.990.950.890.820.890.560.310.750.920.810.950.910.890.720.510.550.510.420.570.410.490.910.920.960.950.880.940.560.460.910.870.900.970.950.710.810.810.870.570.370.800.880.890.790.850.490.620.600.580.500.600.580.500.610.450.330.860.840.740.580.510.390.730.920.910.490.740.960.670.540.850.480.370.880.900.940.820.930.690.490.560.660.430.420.770.730.710.900.990.790.730.900.670.330.610.690.790.730.930.970.910.940.890.490.410.780.780.770.890.990.93

0.920.930.940.970.620.370.850.970.930.920.990.950.890.820.890.560.310.750.920.810.950.910.890.720.510.550.510.420.570.410.490.910.920.960.950.880.940.560.460.910.870.900.970.950.710.810.810.870.570.370.800.880.890.790.850.490.620.600.580.500.600.580.500.610.450.330.860.840.740.580.510.390.730.920.910.490.740.960.670.540.850.480.370.880.900.940.820.930.690.490.560.660.430.420.770.730.710.900.990.790.730.900.670.330.610.690.790.730.930.970.910.940.890.490.410.780.780.770.890.990.93

R

G

B

row

column

11

Slide credit:

Derek

HoiemSlide12

Image filteringCompute a function of the local neighborhood at each pixel in the image

Function specified by a “filter” or mask saying how to combine values from neighborsUses of filtering:Enhance an image (denoise, resize, increase contrast,

etc

)

Extract information (texture, edges, interest points,

etc

)

Detect patterns (template matching)

12

Slide credit:

Kristen

Grauman

, Adapted from Derek

HoiemSlide13

Motivation: noise reduction

Even multiple images of the

same static scene

will not be identical.

13

Slide a

dapted from Kristen

GraumanSlide14

Common types of noise

Salt and pepper noise: random occurrences of black and white pixelsImpulse noise: random occurrences of white pixels

Gaussian noise

: variations in intensity drawn from a Gaussian normal distribution

14

Slide credit:

Steve SeitzSlide15

Gaussian noise

>> noise =

randn(size(im

)).*sigma;

>> output =

im

+ noise;

What is impact of the sigma?

Slide credit:

Kristen

Grauman

Figure from Martial Hebert

15Slide16

Motivation: noise reduction

Even multiple images of the same static scene will not be identical.

How could we reduce the noise, i.e., give an estimate of the true intensities?

What if there’s only one image?

16

Slide credit:

Kristen

GraumanSlide17

First attempt at a solutionLet’s replace each pixel with an average of all the values in its neighborhood

Assumptions: Expect pixels to be like their neighborsExpect noise processes to be independent from pixel to pixel

17

Slide credit:

Kristen

GraumanSlide18

First attempt at a solutionLet’s replace each pixel with an average of all the values in its neighborhood

Moving average in 1D:

18

Slide credit:

S.

MarschnerSlide19

Weighted Moving AverageCan add weights to our moving averageWeights [1, 1, 1, 1, 1] / 5

19

Slide credit:

S.

MarschnerSlide20

Weighted Moving AverageNon-uniform weights [1, 4, 6, 4, 1] / 16

20

Slide credit:

S.

MarschnerSlide21

Moving Average In 2D

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

0

90

90

90

0

0

0

0

0

90

90

90

90

90

00000000000000900000000

0000000000

000000000000000000000009090909090000009090909090000009090909090000009009090900000090909090900000

0

0

0

0

0

0

0

0

0

0

90

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

21

Slide credit:

Steve SeitzSlide22

Moving Average In 2D

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

0

90

90

90

0

0

0

0

0

90

90

90

90

90

00000000000000900000000000

00000000

000000000000000000000009090909090000009090909090000009090909090000009009090900000090909090900000000

0

0

0

0

0

0

0

90

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

22

Slide credit:

Steve SeitzSlide23

Moving Average In 2D

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

0

90

90

90

0

0

0

0

0

90

90

90

90

90

00000000000000900000000000

00000000

000000000000000000000009090909090000009090909090000009090909090000009009090900000090909090900000000

0

0

0

0

0

0

0

90

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

23

Slide credit:

Steve SeitzSlide24

Moving Average In 2D

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

0

90

90

90

0

0

0

0

0

90

90

90

90

90

00000000000000900000000000

0000000010

000000000000000000000009090909090000009090909090000009090909090000009009090900000090909090900000000

0

0

0

0

0

0

0

90

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

24

Slide credit:

Steve SeitzSlide25

Moving Average In 2D

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

0

90

90

90

0

0

0

0

0

90

90

90

90

90

00000000000000900000000000

0000000010

000000000000000000000009090909090000009090909090000009090909090000009009090900000090909090900000000

0

0

0

0

0

0

0

90

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

25

Slide credit:

Steve SeitzSlide26

Moving Average In 2D

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

0

90

90

90

0

0

0

0

0

90

90

90

90

90

00000000000000900000000000

000000001020

000000000000000000000009090909090000009090909090000009090909090000009009090900000090909090900000000

0

0

0

0

0

0

0

90

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

26

Slide credit:

Steve SeitzSlide27

Moving Average In 2D

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

0

90

90

90

0

0

0

0

0

90

90

90

90

90

00000000000000900000000000

000000001020

000000000000000000000009090909090000009090909090000009090909090000009009090900000090909090900000000

0

0

0

0

0

0

0

90

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

27

Slide credit:

Steve SeitzSlide28

Moving Average In 2D

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

0

90

90

90

0

0

0

0

0

90

90

90

90

90

00000000000000900000000000

00000000102030

000000000000000000000009090909090000009090909090000009090909090000009009090900000090909090900000000

0

0

0

0

0

0

0

90

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

28

Slide credit:

Steve SeitzSlide29

Moving Average In 2D

0

10

20

30

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

0

90

90

90

0

0

0

0

0

90

90

90909000000000000000900

000000000000000029Slide credit: Steve SeitzSlide30

Moving Average In 2D

0

10

20

30

30

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

0

90

90

90

0

0

0

0

0

90

9090909000000000000000900

000000000000000030Slide credit: Steve SeitzSlide31

Moving Average In 2D

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

0

90

90

90

0

0

0

0

0

90

90

90

90

90

0000000000000090000000000

0000000001020303030201002040606060402003060909090603003050808090603003050808090603002030505060402010203030303020101010

100000031Slide credit: Steve SeitzSlide32

Correlation filtering

Say the averaging window size is 2k+1 x 2k+1:

Loop over all pixels in neighborhood around image pixel

f[

i,j

]

Attribute uniform weight to each pixel

Now generalize to allow

different weights

depending on neighboring pixel’s relative position:

Non-uniform weights

32

Slide

adapted from Kristen

GraumanSlide33

Correlation filtering

Filtering an image: replace each pixel with a linear combination of its neighbors.The filter “kernel” or “mask” h[

u,v

] is the prescription for the weights in the linear combination.

This is called

cross-correlation

, denoted

33

Slide

adapted from Kristen

GraumanSlide34

Averaging filter

What values belong in the kernel

h

for the moving average example?

0

10

20

30

30

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

0

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

0

0

0

0

0

0

0

0

0

90

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

1

1

1

1

1

1

1

1

1

“box filter”

?

34

Slide

adapted from Kristen

GraumanSlide35

Smoothing by averaging

depicts box filter:

white = high value, black = low value

original

filtered

What if the filter size was 5 x 5 instead of 3 x 3?

35

Slide credit:

Kristen

GraumanSlide36

Boundary issuesWhat about near the edge?the filter window falls off the edge of the image

need to extrapolatemethods:clip filter (black)wrap aroundcopy edgereflect across edge

36

Slide credit:

S.

MarschnerSlide37

Boundary issuesWhat about near the edge?the filter window falls off the edge of the image

need to extrapolatemethods (MATLAB):clip filter (black): imfilter(f, g, 0)wrap around: imfilter(f, g, ‘circular’)copy edge: imfilter(f, g, ‘replicate’)reflect across edge: imfilter(f, g, ‘symmetric’)

37

Slide credit:

S.

MarschnerSlide38

Boundary issuesWhat is the size of the output?MATLAB: output size / “shape” options

shape = ‘full’: output size is sum of sizes of f and gshape = ‘same’: output size is same as f

shape

= ‘valid’: output size is difference of sizes of

f

and

g

f

g

g

g

g

full

f

g

g

g

g

same

f

g

g

g

g

valid

38

Slide credit:

Svetlana

LazebnikSlide39

What if we want nearest neighboring pixels to have the most influence on the output?

Removes high-frequency components from the image (“low-pass filter”).

Gaussian filter

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

90

90

90

90

0

0

0

0

0

90

0

90

90

90

0

0

0

0

0

90

90

90

90

90000000000000

009000000000000000000121242121This kernel is an approximation of a 2d Gaussian function:39Slide credit: Steve SeitzSlide40

Smoothing with a Gaussian

40

Slide credit:

Kristen

GraumanSlide41

Smoothing with a box-filter

41Slide42

Gaussian filters

What parameters matter here?Size of kernel or maskNote, Gaussian function has infinite support, but discrete filters use finite kernels

σ

= 5 with 10 x 10 kernel

σ

= 5 with 30 x 30 kernel

42

Slide credit:

Kristen

GraumanSlide43

Gaussian filters

What parameters matter here?Variance of Gaussian: determines extent of smoothing

σ

= 2 with 30 x 30 kernel

σ

= 5 with 30 x 30 kernel

43

Slide credit:

Kristen

GraumanSlide44

Matlab

>> hsize = 30

;

>> sigma = 5;

>>

h

=

fspecial(‘gaussian

’,

hsize

, sigma);

>>

mesh(h

);

>>

imagesc(h

);

>>

outim

=

imfilter(im

,

h

); % correlation

>>

imshow(outim

);

outim

44

Slide credit:

Kristen

GraumanSlide45

Smoothing with a Gaussian

for sigma=1:3:10 h =

fspecial('gaussian

‘,

hsize

, sigma);

out =

imfilter(im

,

h

);

imshow(out

);

pause;

end

Parameter

σ

is the “scale” / “width” / “spread” of the Gaussian kernel, and controls the amount of smoothing.

45

Slide credit:

Kristen

GraumanSlide46

Properties of smoothing filtersSmoothingValues positive

Sum to 1  constant regions same as inputAmount of smoothing proportional to mask size

Remove “high-frequency” components; “low-pass” filter

46

Slide credit:

Kristen

GraumanSlide47

Filtering an impulse signal

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

a

b

c

d

e

f

g

h

i

What is the result of filtering the impulse signal (image)

f

with the arbitrary kernel

h

?

?

47

Slide

adapted from Kristen

GraumanSlide48

f

h

h

Convolution

Convolution:

Flip the filter in both dimensions (bottom to top, right to left)

Then apply cross-correlation

Notation for convolution operator

48

Slide

adapted from Kristen

GraumanSlide49

Convolution vs. correlation

Convolution

Cross-correlation

For a Gaussian or box filter, how will the outputs differ?

If the input is an impulse signal, how will the outputs differ?

49

Slide

adapted from Kristen

GraumanSlide50

Predict the outputs using correlation filtering

0

0

0

0

1

0

0

0

0

*

= ?

0

0

0

1

0

0

0

0

0

*

= ?

1

1

1

1

1

1

1

1

1

0

0

0

0

2

0

0

0

0

-

*

= ?

50

Slide credit:

Kristen

GraumanSlide51

Practice with linear filters

0

0

0

0

1

0

0

0

0

Original

?

51

Slide credit:

David LoweSlide52

Practice with linear filters

0

0

0

0

1

0

0

0

0

Original

Filtered

(no change)

52

Slide credit:

David LoweSlide53

Practice with linear filters

0

0

0

1

0

0

0

0

0

Original

?

53

Slide credit:

David LoweSlide54

Practice with linear filters

0

0

0

1

0

0

0

0

0

Original

Shifted left

by 1 pixel with correlation

54

Slide credit:

David LoweSlide55

Practice with linear filters

Original

?

1

1

1

1

1

1

1

1

1

55

Slide credit:

David LoweSlide56

Practice with linear filters

Original

1

1

1

1

1

1

1

1

1

Blur (with a

box filter)

56

Slide credit:

David LoweSlide57

Practice with linear filters

Original

1

1

1

1

1

1

1

1

1

0

0

0

0

2

0

0

0

0

-

?

57

Slide credit:

David LoweSlide58

Practice with linear filters

Original

1

1

1

1

1

1

1

1

1

0

0

0

0

2

0

0

0

0

-

Sharpening filter:

accentuates differences with local average

58

Slide credit:

David LoweSlide59

Filtering examples: sharpening

59

Slide credit:

Kristen

GraumanSlide60

Properties of convolutionShift invariant:

Operator behaves the same everywhere, i.e. the value of the output depends on the pattern in the image neighborhood, not the position of the neighborhood.Superposition: h * (f1 + f2) = (h * f1) + (h

* f2)

60

Slide credit:

Kristen

GraumanSlide61

Properties of convolutionCommutative:

f * g = g * fAssociative(f * g) * h = f * (g * h)Distributes over addition f * (g + h) = (f * g) + (f * h)

Scalars factor out

kf * g = f * kg = k(f * g)

Identity:

unit impulse e = […, 0, 0, 1, 0, 0, …]. f * e = f

61

Slide credit:

Kristen

GraumanSlide62

SeparabilityIn some cases, filter is separable, and we can factor into two steps:

Convolve all rowsConvolve all columns62

Slide credit:

Kristen

GraumanSlide63

SeparabilityIn some cases, filter is separable, and we can factor into two steps: e.g.,

What is the computational complexity advantage for a separable filter of size k x k, in terms of number of operations per output pixel?

f

* (

g

*

h

) = (

f

*

g

) *

h

g

h

f

63

Slide credit:

Kristen

GraumanSlide64

Effect of smoothing filters

Additive Gaussian noise

Salt and pepper noise

64

Slide credit:

Kristen

GraumanSlide65

Median filter

No new pixel values introduced

Removes spikes: good for impulse, salt & pepper noise

Non-linear filter

65

Slide credit:

Kristen

GraumanSlide66

Median filter

Salt and pepper noise

Median filtered

Plots of a

col

of the image

Matlab

: output

im

= medfilt2(im, [

h

w

]);

66

Slide

credit: Martial HebertSlide67

Median filterMedian filter is edge preserving

67

Slide credit:

Kristen

GraumanSlide68

Aude Oliva & Antonio Torralba & Philippe G Schyns, SIGGRAPH 2006

Filtering application: Hybrid Images

68

Slide credit:

Kristen

GraumanSlide69

Application: Hybrid Images

Gaussian Filter

Laplacian Filter

Gaussian

unit impulse

Laplacian of Gaussian

69

Slide credit:

Kristen

Grauman

A.

Oliva

, A.

Torralba

, P.G.

Schyns

,

“Hybrid Images,”

SIGGRAPH 2006Slide70

Aude Oliva & Antonio Torralba & Philippe G Schyns, SIGGRAPH 2006

70Slide71

Aude Oliva & Antonio Torralba & Philippe G Schyns, SIGGRAPH 2006

71Slide72

SummaryImage formationImage “noise”Linear filters and convolution useful for

Enhancing images (smoothing, removing noise)Box filterGaussian filterImpact of scale / width of smoothing filterDetecting features (next time)Separable filters more efficient

Median filter: a non-linear filter, edge-preserving

72Slide73

Coming upNext Friday (4/14):

PS0 is due via Canvas, 11:59 PMTuesday:Filtering part 2: filtering for features73Slide74

Questions?See you Tuesday!