22 Outubro 2007 Universidade Federal do Paraná Gerald Dalley Xiaogang Wang and W Eric L Grimson Glasgow Airport 4 cameras 9 video clips 1 for training 8 for testing Dataset Description ID: 488923
Download Presentation The PPT/PDF document "Event Detection Using an Attention-Based..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Event Detection Using an Attention-Based Tracker
22
Outubro 2007. Universidade Federal do Paraná.Gerald Dalley, Xiaogang Wang, and W. Eric L. GrimsonSlide2
Glasgow Airport4 cameras9 video clips1 for training8 for testingDataset Description
2Slide3
LoiteringStaying in the scene for > 1 minuteTheftTaking someone else’s baggageLeft luggageLeaving the scene without your luggage
EvaluationCorrect event timesCorrect 3D locationsWithout false detections
Challenge Problem3Slide4
Basic tracking firstDetect interesting tracksImprove interesting tracks to detect eventsOur Approach
4Slide5
Processing Pipeline5
Tracking & Event Detection
Input Data
Background Subtraction
Background
Modeling
Illumination
NormalizationSlide6
Illumination Changes6
BACKGROUND
126
999
2999
S08Slide7
Illumination Changes by Clip7Slide8
Background Modeling8
Tracking & Event Detection
Input Data
Background Subtraction
Robust Gaussian Fit
Ground-Plane Registration
Background Model Search
Normalized
BACKGROUND
Normalized
S01-S08
Background
Modeling
Background Models
Illumination
NormalizationSlide9
Fundamental assumptionforeground is rare at every pixelReality for PETS 2007…background is rare for the pixels we care about mostSome pixels: foreground as much as 90% of the time
Breaking Adaptive Background Subtraction
9
0
2000
3000
4000Slide10
Robust Gaussian Fit (per pixel)10BACKGROUND clipForeground is rare everywhere
Fit a GaussianRefit to inliersSlide11
Need for Model Adaptation11Another clip (S02)
BACKGROUND’s model: Suboptimal fit
BACKGROUND
’s Gaussian model
Background samples
Foreground samplesSlide12
Model Adaptation12Until convergenceFind inliersShift Gaussian centerSlide13
Improvement13AdaptationIs robustImproves FG/BG classification rates
Before adaptation
After adaptationSlide14
Background Subtraction
14
Tracking & Event Detection
Input Data
Background Subtraction
Background
Modeling
Illumination
Normalization
MRF
blobs
Blob Extraction
Mahalanobis Distance
Background Models
Normalized
S01-S08Slide15
Background Subtraction15
Frame 2000
With Foreground Mask
Foreground Mask
Mahalanobis Distances
Adapted Model
Normalized
-
=
MRFSlide16
Tracking & Event Detection
16
Tracking & Event Detection
Input Data
Background Subtraction
Background
Modeling
Illumination
Normalization
alerts
blobs
Kalman Tracking
Meanshift
Tracking
Event DetectionSlide17
Idea: Focus on tracking what we care about.Loitering humansDropped luggage that becomes dissociated from its ownerKalman trackingConstant velocityLow false positive rate
Blob Tracking
17Slide18
Loitering humansRemain in the scene for a long timeLikely to create isolated tracksDispossessed luggageLikely to create at least an isolated blob detection
Detecting Humans and Luggage
18Object Type
Min. Blob
Area (% of frame)
Max. Blob Area (% of frame)
Min. Blob Track Length
humans
1.5%
3.0%
16s
luggage
0.2%
1.0%
1 frameSlide19
Blob trackingYields high-quality tracks (good)Requires isolated blobs (bad)Meanshift trackerLearn a model (color histogram) from the good blob tracksTracks through occlusions
HumansFind scene entry/exit timesLuggage
Find drop/pickup timesAssociate with human ownersMean-Shift Tracking19Slide20
Results20Slide21
No events occurNone detectedS00 – No Defined Behavior21Slide22
Staged loitering
5.1s late
S01 – General Loitering 1 (Easy)Slide23
Staged loitering1.4s lateS02 – General Loitering 2 (Hard)
23Slide24
Staged loitering8.2s lateDropped luggageShould not trigger an alarmNo alarm triggered
Staged loitering man near purple-outlined woman
MissedUnscripted loiteringDetectedS03 – Bag Swap 1 (Easy)
24Slide25
S04 – Bag Swap 2 (Hard)25
Stay close to each other the whole timeSlide26
S05 – Theft 1 (Easy)26
Victim enters
19.2s late
Luggage stolen
0.08s late
Thief exits
0.08s lateSlide27
S06 – Theft 2 (Hard)27
Thief
Luggage (never isolated)
Victims
Assistant thiefSlide28
S07 – Left Luggage 1 (Easy)28
Luggage dropped
0.12s late
Owner tracked
Luggage taken
0.08s late
By ownerSlide29
S08 – Left Luggage 2 (Hard)29
Owner
drops
: 0.2s late
Owner
picks up
: 0.12s lateSlide30
PETS 2007Problem description, datasets, etc.http://www.pets2007.netMy WebsiteThe paperhttp://people.csail.mit.edu/dalleyg
Websites
30Slide31
Perguntas31Slide32
Illumination Normalization32
whereSlide33
Region-of-Interest Masks33
BG-S04
S06
S05
S07-S08Slide34
Background Subtraction
34
Tracking & Event Detection
Input Data
Background Subtraction
Background
Modeling
Illumination
Normalization
BG-Biased MRF
FG-Biased MRF
Blob Extraction
fragmented blobs
merged blobs
Blob Extraction
Mahalanobis Distance
Background Models
Normalized
S01-S08Slide35
Dual Background Subtraction35Low fragmentation
But blobs mergedGood for human tracking
Mahalanobis Distance Map
Background-Biased Blobs
Foreground-Biased Blobs
Sharp boundaries
But fragmented blobs
Good for dropped luggage detectionSlide36
Tracking & Event Detection
36
Tracking & Event Detection
Input Data
Background Subtraction
Background
Modeling
Illumination
Normalization
alerts
fragmented blobs
merged
blobs
Kalman Tracking
Kalman Tracking
Meanshift
Tracking
Event DetectionSlide37
Luggage: it often doesn’t travel far from the owner, so we need BG-biased to avoid merging the dropped luggage blob with the ownerThe floor is more boring (except for specularities), so camouflaging doesn’t occur much there, relative to the vertical surfacesEasy to tell people fragments from luggage: small people fragments moveThey move when they’re isolated
They move before and after isolationHumansWith the busyness of the scene, a BG-biased MRF produces a lot of fragments and many-to-many blob matching quickly becomes impractical
A FG-biased MRF avoids the fragmentation issue but merges lots of blobsWe only care about loiterersLoiterers are in the scene for a long timeThey’re likely to be isolated from other people at least at some point in timeWhy Dual Trackers/Motion Blobs
37Slide38
Oriented Ellipsoids38