Multispectral Image Analysis for Skin Condition Diagnosis - PowerPoint Presentation

Slide1

Multispectral Image Analysis for Skin Condition Diagnosis

Alison Wright and Yaning Wang

EN.520.483.01.SP20 Bio-Photonics Laboratory

Skin is the largest organ in the human body and consists of two principal layers: the epidermis and the dermis. The epidermis is a stratified squamous epithelium, which consists of 4 types of cells:[3]Melanocytes, Langerhans cells, Merkel cells and Keratinocytes

Malignant melanoma is one of the most rapidly increasing cancers all over the world, with the estimated new cases of 76, 380 and estimated death of 10, 130 in the United States in 2016[1]

Clinical Background and Need

Fig.1&2 Anatomy of the skin and images of in

sinu melanoma(a), invasive melanoma(b)[3]

Dermoscopy is a popular in vivo non-invasive imaging tool that uses polarized light to aid dermatologists in examining pigmented skin lesions based on a set of morphological features. 

Automatically segmenting melanoma from the surrounding    skin is an essential step in computerized analysis of dermoscopic images.

Clinical Background and Need

Fig.3&4Challenges of automated lesion segmentation[1][2]

Fig.5

Dermoscopy

application

Unsupervised color image segmentation[4]Principal Component Analysis (PCA) Thresholding-based methods rely on the histogram distribution of image color which may be altered by hair and bubble in

RoI[1]..Classic GVF : influenced by distractions or noiseAbove methods all employ hand-crafted features that require specialized domain

knowledge

Previous work: automatic skin segmentation

Table.1 Comparison of several unsupervised segmentation methods

Supervised image segmentation method using convolutional neural networks  These models have the capability of learning hierarchical features from raw image data without hand-crafted features.  

Requires thousand annotated training samples

Previous work: automatic skin segmentation

Fig.6 Structure of unet

medical image segmentation[6]

Fig.7 Image mapping from cell membrane to segmentation map[6]

The use of skin color in dermatological diagnosis relies mainly on RGB (red, green, blue) color photography. This approach has limits due to the poor color discrimination of the human eye and brain. Multispectral imaging systems have the capacity to acquire images at different spectral bands, including wavelengths which the human eye is unable to capture.

Previous work: Multispectral imaging

Fig.8&9 Different multispectral imaging system[5][7]

Image preprocessing : hair removal and skin pore removalImage fusion for multispectral skin images based on wavelet decompositionSkin Lesion Segmentation using Bi-Directional

ConvLSTM U-Net Method

Hair removal – Edge detection & region filling Modify image preprocessing method[9]Example from ISIC 2016 dataset

Image preprocessing

Image registration(rigid transform) Image fusion(wavelet based)[8]Dataset:Oliver Lezoray University of Caen, IUT Grand Ouest Normandie, MMI Dpt.

Dermoscopy Skin Lesion Multispectral Image Database30 multispectral dermoscopic images (800 x 600) composed of 6 spectral bands (3 in visible light and 3 in infrared -IR- light)

Image fusion for multispectral system

Fig. 10&11 Structure of image fusion algorithm[7][8]

Image fusion result

Image fusion for multispectral system

near infrared   reference visible light

near infrared   reference visible light

Bi-Directional ConvLSTM U-Net[10]

Network architecture

Replace

Replace

Bi-Directional ConvLSTM U-Net[10]Bi-Directional

ConvLSTM U-NetLoss function: binary_crossentropySoftware: Keras,

Tensorflow-GPU backened 30 epoch batch_size 16Runtime: 40 minutes

Input data/Groundtruth: ISIC 2016 Challenge Part 1 500/200/200 training/validation/testing After image preprocessing

Network architecture

Result : input/

groundtruth/segmented result

Result

Evaluation compared with participants in ISIC 2016 challenge

Result

Participant

Accuracy

Jaccard Index

Sensitivity

Specificity

Urko

Sanchez

0.950

0.832

0.901

0.960

Mahmudur Rahman

0.949

0.884

0.859

0.962

Our

0.932

0.932

0.723

0.991

[1] Y. Yuan, M. Chao and Y. Lo, "Automatic Skin Lesion Segmentation Using Deep Fully Convolutional Networks With Jaccard Distance," in IEEE Transactions on Medical Imaging, vol. 36, no. 9, pp. 1876-1886, Sept. 2017.[2] A. Wong, J.

Scharcanski and P. Fieguth, "Automatic Skin Lesion Segmentation via Iterative Stochastic Region Merging," in IEEE Transactions on Information Technology in Biomedicine, vol. 15, no. 6, pp. 929-936, Nov. 2011.[3] K. Korotkov

and R. Garcia, “Computerized analysis of pigmented skin lesions: A review,” Artif. Intell. Med., vol. 56, no. 2, pp. 69–90, 2012[4]

Hance G, Umbaugh S, Moss R, Stoecker W. Unsupervised color image segmentation: with application to skin tumor borders. IEEE Engineering in Medicine and Biology 1996;15(1):104–11.[5] Romuald

Jolivot, Pierre Vabres, Franck Marzani,Reconstruction of hyperspectral cutaneous data from an artificial neural network-based multispectral imaging system,Computerized

Medical Imaging and Graphics,Volume 35, Issue 2,2011,Pages 85-88,ISSN 0895-6111,[6] Olaf RonnebergerEmail authorPhilipp FischerThomas

Brox, U-Net: Convolutional Networks for Biomedical Image Segmentation,MICCAI 2015 pp 234-241[7] V. Lukinsone, I. Kuzmina, R. Veilande, E. V. Plorina, D. Bliznuks, K. Bolochko, A. Derjabo, I. Lihacova, J. Spigulis, "Multispectral and autofluorescence RGB imaging for skin cancer diagnostics," Proc. SPIE 11065, Saratov Fall Meeting 2018: Optical and Nano-Technologies for Biology and Medicine, 110650A [8] Dong Han, Zhenhua Guo and D. Zhang, "Multispectral palmprint recognition using wavelet-based image fusion," 2008 9th International Conference on Signal Processing, Beijing, 2008, pp. 2074-2077[9] Okuboyejo D, Olugbara O, Odunaike S. Unsupervised restoration of hairoccluded lesion in dermoscopic images. In: Medical Image Understanding and Analysis Conference (MIUA ‘2014); London, UK. 2014. pp. 91-96 [10] R. Azad, M. Asadi, Mahmood Fathy and Sergio Escalera "Bi-Directional

ConvLSTM

U-Net with Densely Connected Convolutions ", ICCV, 2019

Reference

Thank you!