/
Christophe Gauge Christophe Gauge

Christophe Gauge - PowerPoint Presentation

celsa-spraggs
celsa-spraggs . @celsa-spraggs
Follow
420 views
Uploaded On 2016-06-02

Christophe Gauge - PPT Presentation

Gannon University Department of Computer and Information Science Advisor Dr Sreela Sasi Automated Colorization of Grayscale Images Introduction Image Colorization 2 Automated Colorization of Grayscale Images ID: 345845

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Christophe Gauge" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Christophe Gauge

Gannon University

Department of Computer and Information Science

Advisor: Dr. Sreela Sasi

Automated Colorization of Grayscale ImagesSlide2

IntroductionImage Colorization

2

Automated Colorization of Grayscale ImagesSlide3

Introduction (contd.)Digital Image Colorization

3

Automated Colorization of Grayscale ImagesSlide4

Introduction (contd.) Applications of Image Colorization

4

Automated Colorization of Grayscale ImagesSlide5

+

=

+

=

Previous Research

Image

Colorization

5

Automated Colorization of Grayscale ImagesSlide6

Current ResearchProcess Workflow

Texture-based Segmentation

Image

Image

Sample Image

Feature Extraction

Color Descriptors

Texture Descriptors

New Grayscale

Image

Texture-based Segmentation

Feature Extraction

Texture Descriptors

Texture Matching

Colorization Process

Database

6

Automated Colorization of Grayscale ImagesSlide7

Image SegmentationImage segmentation:

Is the partitioning of an image into homogeneous regions based on a set of characteristics.

Is a key element in image analysis and computer vision.

7

Automated Colorization of Grayscale ImagesSlide8

Image Segmentation (contd.)

Clustering:Is one of the methods available for image segmentation.

Is a process which can be used for classifying pixels based on similarity according to the pixel’s color or gray-level intensity.

8

Automated Colorization of Grayscale ImagesSlide9

Image Segmentation (contd.)Despite the substantial amount of research performed to date, the design of a robust and efficient clustering algorithm remains a very challenging problem

9

Automated Colorization of Grayscale ImagesSlide10

Color-based Image SegmentationComposite Image

10

Automated Colorization of Grayscale ImagesSlide11

Color-based Image Segmentation

Composite

Image

with salt & pepper noise added

11

Automated Colorization of Grayscale ImagesSlide12

Texture-based Image Segmentation

12

Automated Colorization of Grayscale ImagesSlide13

Workflow ProcessTexture-Based Image Segmentation

Original Image

Filtered Image

Filtered Image

Filtered Image

Filtered Image

Feature Image

Feature Image

Feature Image

Feature Image

Feature Image

Blobs

Gabor Filters

Energy Computation

Segmentation

Add, mean smoothing, normalization

13

Automated Colorization of Grayscale ImagesSlide14

14

Automated Colorization of Grayscale Images

Image Segmentation

Multi-Channel Filtering - Gabor TransformSlide15

Previous Research

(contd.)

Texture-Based Segmentation

15

Automated Colorization of Grayscale ImagesSlide16

16

Automated Colorization of Grayscale Images

Image Segmentation

Normalized Sum of Gabor ResponsesSlide17

Current ResearchProcess Workflow

Texture-based Segmentation

Image

Image

Sample Image

Feature Extraction

Color Descriptors

Texture Descriptors

New Grayscale

Image

Texture-based Segmentation

Feature Extraction

Texture Descriptors

Texture Matching

Colorization Process

Database

17

Automated Colorization of Grayscale ImagesSlide18

Previous Research

(contd.)

Clustering and Feature Extraction

18

Automated Colorization of Grayscale ImagesSlide19

Previous ResearchThe K-means algorithm has been used for a fast and crisp “hard” segmentation.

The Fuzzy set theory has improved this process by allowing the concept of partial membership, in which an image pixel can belong to multiple clusters.

This “soft” clustering allows for a more precise computation of the cluster membership, and has been used successfully for image clustering and segmentation.

19

Automated Colorization of Grayscale ImagesSlide20

The Fuzzy C-means clustering (FCM) algorithm [1] is a widely used method for “soft” image clustering. However, the FCM algorithm is computationally intensive.

It is also very sensitive to noise because it only iteratively compares the properties of each individual pixel to each cluster in the feature domain.

Previous Research

(contd.)

[1]

James C.

Bezdek

, Pattern Recognition with Fuzzy Objective Function Algorithms. New York: Plenum, 1981.

20

Automated Colorization of Grayscale ImagesSlide21

Image Segmentation

Modified Fuzzy C-means Clustering

21

Automated Colorization of Grayscale ImagesSlide22

Previous Research (contd.)Fuzzy C-means clustering (FCM) Algorithm

22

Automated Colorization of Grayscale ImagesSlide23

Previous Research (contd.)FCM Pseudo-code

Step 1 Set the number c of clusters, the fuzzy parameter m, and the stopping condition εStep 2 Initialize the fuzzy membership values µ

Step 3 Set the loop counter b

= 0Step 4 Calculate the cluster centroid

values using (3)Step 5 For each pixel, compute the membership values using (4) for each clusterStep 6 Compute the objective function

A. If the value of A between consecutive iterations < ε then stop, otherwise set b=

b+1 and go to step 4

23Automated Colorization of Grayscale ImagesSlide24

[2]

Stelios

Krinidis

and Vassilios Chatzis

, "A Robust Fuzzy Local Information C-means Clustering Algorithm," Image Processing, IEEE Transactions on, pp. 1-1, 2010.

Previous Research

(contd.)Modified Fuzzy C-means clustering with Gki

factorIn order to improve the tolerance to noise of the Fuzzy C-means clustering algorithm, Krinidis and

Chatzis [2] have proposed a new Robust Fuzzy Local Information C-means Clustering (FLICM) algorithm by introducing the novel Gki factor. The purpose of this algorithm is to adjust the fuzzy membership of each pixel by adding local information from the membership of neighboring pixels.

24Automated Colorization of Grayscale ImagesSlide25

Previous Research (contd.)Modified Fuzzy C-means clustering with Gki

factor

Sliding window of size 1 around the

i

th

pixel

The

G

ki

factor is obtained by using a sliding window of predefined dimensions:25Automated Colorization of Grayscale ImagesSlide26

Previous Research (contd.)Modified Fuzzy C-means clustering with Gki

factor

 The G

ki factor is calculated by using the following equation:

26

Automated Colorization of Grayscale ImagesSlide27

Current AlgorithmModified Fuzzy C-means clustering with novel Hik factor

 

This algorithm is further improved by including both the local spatial information from neighboring pixels and the spatial Euclidian distance of each pixel to the cluster’s center of gravity.

In this research, the algorithm is also extended for clustering of color images in the Red-Green-Blue (RGB) color space.

27

Automated Colorization of Grayscale ImagesSlide28

Current Algorithm (contd.)

Illustration of the new

H

ik

factor displaying the spatial Euclidian distance to the center of gravity of each cluster

28

Automated Colorization of Grayscale ImagesSlide29

Current Algorithm (contd.)Process Workflow

Customize Parameters

Calculate cluster membership values

Compute

G

ki

Readjust membership values

Compute

H

ki

Compute objective functionDefuzzification and clustering

-

Image

Calculate cluster

centroid

29

Automated Colorization of Grayscale ImagesSlide30

Current Algorithm (contd.)Modified Fuzzy C-means Clustering

30

Automated Colorization of Grayscale ImagesSlide31

Simulation and Results

Synthetic Grayscale Test Image

31

Automated Colorization of Grayscale ImagesSlide32

Natural test image

FCM segmentation

with 5 clusters

FCM segmentation

using the modified FCM algorithm

with 5 clusters,

G

ki

window=1 and

H

ik

Simulation and Results

Natural Test Image

32

Automated Colorization of Grayscale ImagesSlide33

Simulation and Results

Synthetic Grayscale Test Image

Synthetic 4-color test image

with added salt and pepper noise

FCM clustering

FCM clustering

with

G

ki

window=1 and with

H

ik

FCM clustering

with

G

ki

window=5 and with

H

ik

33

Automated Colorization of Grayscale ImagesSlide34

Synthetic 4-color test image

with added salt and pepper noise

FCM clustering

FCM clustering

with

G

ki

window=1 and with

H

ik

FCM clustering

with

G

ki

window=5 and with

H

ik

Simulation and Results

Synthetic Color Test Image

34

Automated Colorization of Grayscale ImagesSlide35

Image SegmentationClustering Demo

35

Automated Colorization of Grayscale ImagesSlide36

Modified Fuzzy C-means Clustering

Summary

In this research, the FCM with the

G

ki

factor is modified using the

H

ik

factor, and the algorithm is extended for the clustering of color images. The use of the sliding window in the

Gki factor improves the segmentation results by incorporating local information about neighboring pixels in the membership function of the clusters. However, this resulted in a significant increase in the number of calculations required for each iteration for each pixel, and can be given by

36Automated Colorization of Grayscale ImagesSlide37

Modified Fuzzy C-means Clustering

Summary (contd.)

By combining the

G

ki

and the

H

ik

factors, this modified FCM algorithm considerably reduced the number of iterations needed to achieve convergence. The tolerance to noise of the Fuzzy C-means algorithm is also greatly increased, allowing for an improved capability to obtain coherent and contiguous segments from the original image.

37Automated Colorization of Grayscale ImagesSlide38

Modified Fuzzy C-means Clustering

Summary (contd.)

However, because of the radial nature of the spatial Euclidean distance to the cluster’s center of gravity, this new method may not be as effective for images containing circular shapes, or for images where the cluster’s center of gravity are close to each-other.

In this research, the FCM is extended for the clustering of color images in the RGB color space. The effectiveness of this algorithm may be tested for images in other color spaces also.

38

Automated Colorization of Grayscale ImagesSlide39

Current ResearchProcess Workflow

Texture-based Segmentation

Image

Image

Sample Image

Feature Extraction

Color Descriptors

Texture Descriptors

New Grayscale

Image

Texture-based Segmentation

Feature Extraction

Texture Descriptors

Texture Matching

Colorization Process

Database

39

Automated Colorization of Grayscale ImagesSlide40

40Automated Colorization of Grayscale Images

Sample Color ImagesSlide41

41

Automated Colorization of Grayscale Images

Image Segmentation

Normalized Sum of Gabor ResponsesSlide42

Image Segmentation

Feature Extraction

42

Automated Colorization of Grayscale ImagesSlide43

Image Segmentation

Feature Extraction

(contd.)

43

Automated Colorization of Grayscale Images

Blob Filtering for color and texture extraction.Slide44

44

Automated Colorization of Grayscale Images

Texture and Color database

Image Segmentation

Feature

Extraction

(contd.)Slide45

45Automated Colorization of Grayscale Images

Current Research

Process Workflow

Texture-based Segmentation

Image

Image

Sample Image

Feature Extraction

Color Descriptors

Texture Descriptors

New Grayscale

Image

Texture-based Segmentation

Feature Extraction

Texture Descriptors

Texture Matching

Colorization Process

DatabaseSlide46

46Automated Colorization of Grayscale Images

Grayscale Image ProcessingSlide47

47Automated Colorization of Grayscale Images

Current Research

Process Workflow

Texture-based Segmentation

Image

Image

Sample Image

Feature Extraction

Color Descriptors

Texture Descriptors

New Grayscale

Image

Texture-based Segmentation

Feature Extraction

Texture Descriptors

Texture Matching

Colorization Process

DatabaseSlide48

48Automated Colorization of Grayscale Images

Previous Research

Visual descriptors

Visual

descriptors are descriptions of the visual features of the contents

of images.They

describe elementary characteristics such as the shape, color, and texture.

MPEG-7 is a multimedia content description standard. It was standardized in ISO/IEC 15938 (Multimedia content description interface

).This description is associated with the content itself, to allow fast and efficient searching for material that is of interest to the user.

MPEG-7 is formally called Multimedia Content Description Interface. Thus, it is not a standard which deals with the actual encoding of moving pictures and audio, like MPEG-1, MPEG-2 and MPEG-4. It uses XML to store

metadata.Slide49

49Automated Colorization of Grayscale Images

Previous Research

Visual descriptors

http://chatzichristofis.info/?page_id=213

The

Img(Rummager) application was developed in the Automatic Control Systems & Robotics Laboratory at the Democritus University of Thrace-Greece

.The application can execute an image search based on a query image, either from XML-based index files, or directly from a folder containing image files, extracting the comparison features in real time. Slide50

Previous Research

(contd.)

Content-Based Image Retrieval

50

Automated Colorization of Grayscale ImagesSlide51

MPEG-7 EHD

Fuzzy Spatial BTDH

ADS

51

Automated Colorization of Grayscale Images

Previous Research

(contd.)

Content-Based Image RetrievalSlide52

Image Descriptors used:MPEG-7 Homogeneous Texture Descriptor: Edge Histogram

 Descriptor (EHD). 

CCD for Medical Radiology Images: Brightness and Texture Directionality Histogram (BTDH)Fuzzy rule based scalable composite descriptor (BTDH) is a compact composite descriptor that can be used for the indexing and retrieval of radiology medical images. This descriptor uses brightness and texture characteristics as well as the spatial distribution of these characteristics in one compact 1D vector. The most important characteristic of the proposed descriptor is that its size adapts according to the storage capabilities of the application that is using it. This characteristic renders the descriptor appropriate for use in large medical (or gray scale) image databases.

Simulation Results

(contd.)

Content-Based Image Retrieval (CBIR)

52

Automated Colorization of Grayscale ImagesSlide53

Simulation Results

(contd.)

Content-Based Image Retrieval (CBIR)

53

Automated Colorization of Grayscale ImagesSlide54

54Automated Colorization of Grayscale Images

Current Research

Process Workflow

Texture-based Segmentation

Image

Image

Sample Image

Feature Extraction

Color Descriptors

Texture Descriptors

New Grayscale

Image

Texture-based Segmentation

Feature Extraction

Texture Descriptors

Texture Matching

Colorization Process

DatabaseSlide55

The RGB color space is defined by the three chromaticities of the red, green, and blue additive primaries, and can produce any chromaticity that is the triangle defined by those primary colors.

The YCbCr

color space is used in video and digital photography systems.

Y is the luma (luminance ) component and

Cb and Cr are the blue-difference and red-difference chroma

components.

Simulation Results (contd.)

Image Colorization

55

Automated Colorization of Grayscale ImagesSlide56

56

Automated Colorization of Grayscale Images

Image from Wikipedia

Simulation Results

(contd.)

Image ColorizationSlide57

Simulation Results

(contd.)

Colorization

57

Automated Colorization of Grayscale ImagesSlide58

Conclusion and Future Work

New and innovative method

Automating example-based colorization

Combines several state-of-the-art techniques

Reasonably accurate results were obtained

Several of the steps require custom parameters

computationally very intensive

Texture retrieval needs improvement

Complex textures containing multiple colors

Anisotropic diffusion for preserving strong edge informationCombining these techniques in order to automatically colorize grayscale images is a viable option

58Automated Colorization of Grayscale ImagesSlide59

Conclusion and Future Work (contd.)

I

mages segmentation and clustering methods computationally

very intensive, P

rocessing

time for each 600x450 sample color image

took 20

minutes on a quad-core Intel 2.6 GHz processor

.Texture retrieval methods still need to be improved for scale and rotation invarianceStore

more complete color descriptors to accommodate more complex textures containing multiple colors. Anisotropic diffusion could also be used to smooth the Gabor response images while preserving strong edge information.

Testing conducted as part of this research proved that the ability to combine these techniques in order to automatically colorize grayscale images is a viable option. 59Automated Colorization of Grayscale ImagesSlide60

References

[1]

Anat

Levin,

Dani

Lischinski

, and

Yair

Weiss, "Colorization using optimization,"

ACM Transactions on Graphics, vol. 23, no. 3, p. 689–694, 2004.[2]R. Irony, D. Cohen-Or, and D. Lischinski, "Colorization by example," in

Eurographics Symposium on Rendering, 2005, p. 277–280.[3]

Ashikhmin M., Mueller K. Welsh T., "Transferring Color to Greyscale Images,".

[4]

X., Wan L., Qu Y., Wong T., Lin S., Leung C., Heng P. Liu, "Intrinsic colorization,"

ACM Trans. Graph.

, vol. 27, no. 5, p. 152, 2008.

[5]

Malik J. Perona P., "Preattentive texture discrimination with early vision mechanisms,"

J. Opt. Soc. Am. A

, vol. 7, no. 5, May 1990.

[6]

A. K. Jain and F.

Farrokhnia

, "Unsupervised texture segmentation using

Gabor filters

,"

Pattern Recognition

, vol. 24, no. 12, pp. 1167-1186, 1991.

[7]

Seo

Naotoshi

, "Texture Segmentation using Gabor Filters," University of Maryland, College Park, MD, Project ENEE731 , 2006.

[8]

Xiaoming

Hu

,

Xinghui

Dong,

Jiahua

Wu, Ping

Zou

Junyu

Dong, "Texture Segmentation Based on Probabilistic Index Maps," in

International Conference on Education Technology and Computer

, 2009, pp. 35-39

.

60

Automated Colorization of Grayscale ImagesSlide61

References (contd.)

[9]

X Muñoz, J

Freixeneta

, X

Cufı́a

, and J

Martı́a

, "Strategies for image segmentation combining region and boundary information,"

Pattern Recognition Letters, vol. 24, no. 1-3, pp. 375-392, January 2003.

[10]James C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algorithms. New York: Plenum, 1981.[11]

Chuang Keh-Shih, Tzenga Hong-Long, Chen Sharon, Wu Jay, and Chen Tzong-Jer, "Fuzzy c-means clustering with spatial information for image segmentation," Computerized Medical Imaging and Graphics, vol. 30, no. 1, pp. 9-15, January 2006.[12]

Zhou Huiyu, Schaefer Gerald, Sadka

Abdul H., and

Celebi

M.

Emre

, "Anisotropic Mean Shift Based Fuzzy C-Means Segmentation of

Dermoscopy

Images,"

IEEE Journal of Selected Topics in Signal Processing

, vol. 3, no. 1, pp. 26-34, February 2009.

[13]

Stelios

Krinidis

and

Vassilios

Chatzis

, "A Robust Fuzzy Local Information C-means Clustering Algorithm,"

Image Processing, IEEE Transactions on

, pp. 1-1, 2010.

[14]

Gauge

Christophe

and

Sasi

Sreela

,

"Automated Colorization of

Grayscale

Images Using Texture Descriptors and a Modified Fuzzy C-Means Clustering,“

Journal of Intelligent Learning Systems and Applications (JILSA), Vol. 4 No. 2, 2012, pp. 135-143, DOI: 10.4236/

jilsa

.

61

Automated Colorization of Grayscale ImagesSlide62

62Automated Colorization of Grayscale Images

Questions?