/
Partial Face Recognition Partial Face Recognition

Partial Face Recognition - PowerPoint Presentation

liane-varnes
liane-varnes . @liane-varnes
Follow
423 views
Uploaded On 2016-04-11

Partial Face Recognition - PPT Presentation

S Liao A K Jain and S Z Li Partial Face Recognition AlignmentFree Approach  IEEE Transactions on Pattern Analysis and Machine Intelligence Vol 35 No 5 pp 11931205 May 2013 ID: 279029

image face keypoint partial face image partial keypoint recognition src faces gallery mkd images keypoints probe descriptors subjects experiment

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Partial Face Recognition" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Partial Face Recognition

S. Liao, A. K. Jain, and S. Z. Li, "Partial Face Recognition: Alignment-Free Approach", 

IEEE Transactions on Pattern Analysis and Machine Intelligence

, Vol. 35, No. 5, pp. 1193-1205, May 2013,

doi

: 10.1109/TPAMI.2012.191Slide2

Cooperative Face Recognition

People stand

in front of a camera with good illumination conditions.

Border pass, access control, attendance, etc.Slide3

Unconstrained Face Recognition

Images

are captured

with less user cooperation, in more challenging conditionsVideo surveillance, hand held system, etc.Slide4

Partial Faces in Unconstrained EnvironmentsSlide5

Face Recognition and the London Riots

Summer 2011

Widespread looting and rioting:

Extensive CCTV Network:

FR lead to

many arrests:

Yet, many suspects still unable to be identified by COTS FRS:Slide6

Face Detection in a Crowd

Normalized Pixel Difference (NPD) Face Detector

OpenCV

Viola-Jones Face Detector

PittPatt-5 Face DetectorSlide7

Unconstrained Face Recognition

Problem:

Recognize an arbitrary face image captured in unconstrained environment

Possible areas for improvement:

Face detection?Alignment?Feature representation?Classification?

Importance:

Recognize a suspect in crowd

Identify a face from its partial imageSlide8

Alignment Free

Partial Face Recognition (PFR)

Proposed alignment-free method: MKD-SRCSlide9

Alignment Free

Partial Face Recognition (PFR)

Multi

Keypoint

Descriptors (MKD)Each image is described by a set of keypoints and descriptors (e.g. SIFT):Keypoints:

p

1

,

p

2

, …, pk

Descriptors:

d

1

,

d

2

, …,

d

k

The number of descriptors,

k

, may be different from image to imageSlide10

Alignment Free

Partial Face Recognition (PFR)Slide11

Sparse Representation Classification (SRC) based on MKD

Descriptors

from the

same class

c can be viewed as a sub-dictionary:Combining sub-dictionaries: For each descriptor

y

i

of

in a probe image, solve

Determine the identity of the probe image by SRC:Slide12

Sparse Representation Classification (SRC) based on MKDSlide13

An Example Solution

MKD-SRC is more

discriminant

for PFR

The horizontal axis represents the index of the gallery

keypoint

descriptors

The vertical axis denotes the coefficient strength, as computed by

Morgan Freeman

Quincy Delight JonesSlide14

Large Scale Partial Face Recognition

In the dictionary, the number of atoms,

K

, can be of the order of

millionsFast atom filtering:

(*)

For each

y

i

, we filter out only

T

(

T

<<

K

) atoms according to the top

T

largest values in

c

i

, resulting in a small sub-dictionary.

The computation of Eq. (*) is linear

w.r.t

. K, the selection of the largest T

values can be done in O(K), thus the proposed fast atom filtering scales linearly w.r.t.

K, while the remaining computation of l1 minimization takes a constant time.Slide15

Effects of the Fast Atom Filtering

A subset of FRGCv2, with 1,398 gallery images and 466 probe images, resulting in K=111,643 for the dictionary.Slide16

Keypoint Descriptors

Scale Invariant Feature Transform (SIFT)

Advantage: promising results, efficient to compute

Disadvantage: limited number of

keypoints (~80), not affine invariantGabor Ternary Pattern (GTP) descriptor

Adopts edge based affine invariant

keypoint

detector called

CanAff

, which provides sufficient number of keypoints (~800) for PFR

Robust to illumination variations and noisesEven with fast atom filtering, run time is O(n

2

) with

keypoints

per image

10 times more

keypoints

, 100 times slowerSlide17

Keypoint Descriptors

SIFT

(37)

GTP

(first 150 of 571)Slide18

GTP DescriptorSlide19

Normalize the detected region to 40x40 pixels

Clipped Z-Score normalization:

Normalize the pixel values to [0,1]

Reduce the influence of illumination variation

Reduce the influence of extreme pixel values

Keypoint

Region NormalizationSlide20

Gabor Filters

Odd Gabor filters with small scale, 4 orientations

Imaginary part of Gabor filters, sensitive to edges and their locations.

Scale 0, 5x5 support area, 0

º, 45º, 90

º,

135

ºSlide21

Encode the responses of the 4 Gabor filters

Local structure about the responses of Gabor filters in 4 orientations

Examples of some local structures encoded

4 orientations

Local Ternary Pattern

2201 2011 0222 Slide22

Building the descriptor

Calculate the histogram of local ternary patterns (3

4

bins) over each grid cell, and concatenate them to form a 1,296 element

vectorTransform by a sigmoid function ( tanh(20x

) )

Reduce the influence of extreme values

Reduce the dimension to 128 by PCA Slide23

GTP Descriptor

Local patch of 40x40 pixels

4x4 grid cells

3

4

bins for each cell

1296 bins in total

PCA to 128 dimsSlide24

Labeled Faces in the Wild (LFW)

1

Real faces from the internet, most with non-frontal views or occlusion

13,233 images of 5,749 subjects

1

http://vis-www.cs.umass.edu/lfw/Slide25

Experiments on LFW

MKD-SRC performs better than

FaceVACS

, but is not as good as

PittPatt

Fusion of MKD-SRC &

PittPatt

improves performanceSlide26

Experiments on LFW

Face image pairs that can be correctly recognized by MKD-SRC but not by

PittPatt

at FAR=1%Slide27

Experiment on

PubFig

Database

2

Large-scale open-set identification

Gallery: 5,083 full frontal faces

Probe:

817 partial

faces (belong to gallery) with large

pose variation

or occlusion

7,210 faces as impostors (do not belong

to gallery)

2

http://www.cs.columbia.edu/CAVE/databases/pubfig/Slide28

Experiment on PubFig

Database

Proposed MKD-SRC method is better than two commercial SDKs,

FaceVACS

and PittPattSlide29

Synthetic Partial Face Image Generation

5/28/2013

29

Rotate images;

degree of rotation

randomly drawn

from

a normal distribution (mean 0, std. dev. 10º)

Sample width and height for the patch, drawn from a uniform distribution from 50-100% of original size

Sample a starting position for the patch

Randomly rescale the patch

Rotated

(size reduced for display)

Original size patch

Rescaled patch

Original

(size reduced for display)Slide30

FRGC+ Dataset

Open set recognition

FRGC dataset

Gallery:

466 FRGC Images

10,000 PCSO Images

Probe

A. 15,562 FRGC partial faces (matching the FRGC subjects in gallery)

B. 10,000 PCSO partial faces (not matching any gallery subjects)

Average time per probe image ~1 second vs. 10,466 image gallery

Pittpatt

5.2 fails to enroll ~50% of the partial facesSlide31

Experiment on MOBIO database

3

Videos captured by mobile phone from six universities/institutes in Europe

4,880 videos of 61 subjects for verification

Gallery (top) and probe (bottom)

3

http://www.idiap.ch/dataset/mobioSlide32

32

Experiment on MOBIO database

A. Female

B. MaleSlide33

Experiment on the Mobile dataset

Unconstrained face images with a mobile phone

Pose, illumination, expression, occlusion or invisible parts

Gallery images of 14 subjects plus additional 1,000 background subjects; one image/subject

Probe: 168 mobile phone images of 14 subjects, with additional 1,000 impostors

Open-set (watch-list) identification experimentSlide34

PittPatt

cannot be applied because the probe faces cannot be aligned

Experiment on the Mobile datasetSlide35

Other Keypoint

Matching Methods

Keypoint

based representations are naturally variable size

The previously discussed method reconstructs each probe keypoint from the gallery using SRCOther options:

Bag of words methods – fixed sized representation over a dictionary

Modified

Hausdorff

Distance – apply a general distance metric to sets of pointsSlide36

Modified Hausdorff

Distance

Given a distance metric d, and 2 sets of

keypoints

A and B find:D(A,B) = mean(mina in A(d(a,B

)))

Compute the min distance from each

keypoint

in A to a

keypoint in B, average the results over all keypoints

in AD(A,B) ≠ D(B,A)MHD(A,B) = max(D(A,B), D(B,A))We calculate all probe to gallery

keypoint

distances for the atom filtering step, so computing MHD is not costlySlide37

Summary

Face recognition based on applying SRC to local

keypoint

descriptorsOutperformed by other methods for

mugshot style images, but can be used even when faces cannot be aligned E.g. only part of the face is available, or face/eye detection fail