Person-Specific Domain Adaptation with Applications to - PowerPoint Presentation

Slide1

Person-Specific Domain Adaptation with Applications to Heterogeneous Face Recognition (HFR)

Presenter: Yao-Hung Tsai Dept. of Electrical Engineering, NTU

Oral Presentation:

2014.05.02Slide2

OutlineFace RecognitionConventional ApproachHeterogeneous Face Recognition

Domain Adaptation ApproachProposed ApproachDomain-independent Component AnalysisPerson-specific ClassifierCombinational FrameworkExperimentsSlide3

OutlineFace RecognitionConventional Approach

Heterogeneous Face RecognitionDomain Adaptation ApproachProposed ApproachDomain-independent Component AnalysisPerson-specific ClassifierCombinational FrameworkExperimentsSlide4

Heterogeneous Face RecognitionFace Recognition – Face IdentificationIdentify the subject from the captured images

Slide5

Heterogeneous Face RecognitionFace Recognition – Face VerificationVerify a specific subject with respect to the captured image Slide6

Heterogeneous Face RecognitionFace Recognition Application

Access Control System

Photo auto-tagging

Crime investigation

……Slide7

OutlineFace RecognitionConventional Approach

Heterogeneous Face RecognitionDomain Adaptation ApproachProposed ApproachDomain-independent Component AnalysisPerson-specific ClassifierCombinational FrameworkExperimentsSlide8

Conventional ApproachDirect methodDirect compare two images based on their

pixel values v.s. Advantages :Naïve, simple to implementDisadvantagesRequire lots of computation effortSlide9

Conventional ApproachA common

method : Eigenface methodRepresentation: pixel intensityCollecting several images as the training set:Then we apply PCA to this set.

=

…

n

dSlide10

Conventional ApproachPCAPCA projects columns of X from high-dimension ( ) to low dimension ( ).PCA make projection variance maximized by optimize

:After solving the optimization we will get a set of basis vectors (faces):We can reconstruct the images by:

Ex: 2 dim to 1 dim

Note: v

1

will capture most data varianceSlide11

Conventional ApproachThe combinational coefficients will be the new feature of face:

For recognition, we simply project all images into this k-dimensional space and apply classifiers.

Note: Same class cluster together.Slide12

Conventional ApproachHowever, there

exist several problemsTraditional pattern recognition problems typically deal with Training and test data collected from the same feature spaceIn real word applications, these data are Collected from different feature domainsExhibiting distinct feature distributionsWe call this cross-domain recognition problemsAlso called Heterogeneous Face Recognition (HFR)Slide13

OutlineFace RecognitionConventional Approach

Heterogeneous Face RecognitionDomain Adaptation ApproachProposed ApproachDomain-independent Component AnalysisPerson-specific ClassifierCombinational FrameworkExperimentsSlide14

Heterogeneous Face RecognitionHFR is an emerging task in biometrics

Sketches

in Criminal Cases

Night

Vision

CameraSlide15

Heterogeneous Face RecognitionFace recognition conduct on different domainsSlide16

Heterogeneous Face RecognitionWhen conventional FR meets HFR …If directly apply PCA on images cross domains (

e.x. infra-red v.s. visible spectrum)We visualize the data distribution of first 3 dimensions :

VIS

NIR

VIS

Domain

NIR

DomainSlide17

Heterogeneous Face RecognitionObserving the difference between domains :Instances with same class are far from each others.

Data from same domain close to each others.That is,Domain difference dominates the data variance.So, we need to conduct domain adaptation approachFor comparing images from source and target domain

andSlide18

OutlineFace RecognitionConventional Approach

Heterogeneous Face RecognitionDomain Adaptation ApproachProposed ApproachDomain-independent Component AnalysisPerson-specific ClassifierCombinational FrameworkExperimentsSlide19

Domain Adaptation ApproachThere are numerous approaches of domain adaptation

Observing domain invariant featuresLocal Binary Patterns (LBP) – PAMI 2006Projecting images on common feature spaceCanonical Correlation Analysis (CCA)Partial Least Squares (PLS) – CVPR 2011Semi-coupled Dictionary Learning (SCDL) – CVPR 2012Coupled Dictionary Learning (CDL) – ICCV 2013Match distributions between cross domain imagesMatch marginal distributions (TCA) – TNN 2011Match also joint distributions (JDA) – ICCV 2014Slide20

Domain Adaptation ApproachIllustrate the notation of external dataTake access control system (ACS) as an example

At first, we usually cannot get the user’s images in advanceThus, we need to use images from other subjects collected in advance to model the systemLet us call it external dataNote: Images from both domains need to be collected.

External

Data

…

…Slide21

Domain Adaptation ApproachMost of the approaches require

a large number of paired external dataHowever, it is very difficult to

collect them !Collecting external data with no labeled information is much easierMoreover, direct use of external data might be non-preferable

There’s no guarantee of the same feature distribution among external data and test data

For

example,

the

common feature space observed from the face images of females will

not generalize

well to those of males. Slide22

Domain Adaptation ApproachSo, I proposed an approach with the following propertiesRequire no labeled information in external data

Advocate the learning of person-specific domain adaptation model for HFRDiCA ( Domain-independent Components Analysis) is proposed to build a common feature spaceSlide23

OutlineFace RecognitionConventional Approach

Heterogeneous Face RecognitionDomain Adaptation ApproachProposed ApproachDomain-independent Component AnalysisPerson-specific ClassifierCombinational FrameworkExperimentsSlide24

Domain-independent Component AnalysisReview the observations in HFR problems

Domain difference dominates the data variance.We check differences of projected means (MMD) for every dimensions of PCA space:

and

…

: mean of NIR

: mean of VIS

MMD:

-Slide25

Domain-independent Component AnalysisThen we can discard the components with high MMD value.

We get the final domain-independent projection matrix:

External

Data

…

…

Domain-independent Components Analysis:

DiCASlide26

Domain-independent Component AnalysisSo far, we can directly project user’s images to DiCA

space and match test images.However, to address the issue that subjects from external data are different from users and to improve the performance.I further proposedPerson-specific Classifier (PC)Slide27

OutlineFace RecognitionConventional Approach

Heterogeneous Face RecognitionDomain Adaptation ApproachProposed ApproachDomain-independent Component AnalysisPerson-specific ClassifierCombinational FrameworkExperimentsSlide28

Person-specific ClassifierForming a specific classifier for the input test image, for this specific classifier outperforms than the general

oneSVM (support vector machine) classifier is chose to be this person-specific classifierChoose test data as positive instance.User defined negative instances could be chosen for different usage.

Positive

Negative

Person-specific classifier

Slide29

Person-specific ClassifierSupport Vector Machines (SVM)Classifier to discriminate two categories

dataTraining dataset xi ∈ A+ ⇔ yi = 1 & xi ∈ A- ⇔ yi = -1 Slide30

Person-specific ClassifierGoal : Predict the unseen class label for new

dataFind a function f : Rn → R by learning from data f(x) ≥ 0 ⇒x ∈ A+ and f(x) < 0 ⇒ x ∈ A-Simplest function is linear : f (x) = w⊤

x + b Slide31

OutlineFace RecognitionConventional Approach

Heterogeneous Face RecognitionDomain Adaptation ApproachProposed ApproachDomain-independent Component AnalysisPerson-specific ClassifierCombinational FrameworkExperimentsSlide32

Combinational Framework

External

Data

…

…

NIR Data

VIS Data

User’s images

…

Test

positive

negative

Similarity Score

User’s images

Form DiCA

Subspace

NIR

VISSlide33

OutlineFace RecognitionConventional Approach

Heterogeneous Face RecognitionDomain Adaptation ApproachProposed ApproachDomain-independent Component AnalysisPerson-specific ClassifierCombinational FrameworkExperimentsSlide34

ExperimentsTwo HFR scenario:

Photo – sketch (CUHK database)VIS – NIR (CASIA 2.0 database)Identification TaskFor photo-sketch, there are 100 gallery images and 100 test images.For VIS-NIR, there are 359 gallery images and 6200 test images (with different occlusions)Slide35

ExperimentsTwo HFR scenario:

Photo – sketch (CUHK database)VIS – NIR (CASIA 2.0 database)Identification TaskFor photo-sketch, there are 100 gallery images and 100 test images.For VIS-NIR, there are 359 gallery images and 6200 test images (with different occlusions)Slide36

ExperimentsSketch-to-photo Dataset

NIR-to-VIS DatasetSlide37

The EndThank You!