MultiLabel MRF Optimization David Tsai Matthew Flagg James M Rehg Computational Perception Lab School of Interactive Computing Georgia Institute of Technology Tracking Animals for Behavior Analysis ID: 405478
Download Presentation The PPT/PDF document "Motion Coherent Tracking with" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Motion Coherent Tracking with Multi-Label MRF Optimization
David Tsai Matthew Flagg James M. RehgComputational Perception LabSchool of Interactive ComputingGeorgia Institute of TechnologySlide2
Tracking Animals for Behavior Analysis
Wide range of animal morphologies and biological questions“Tracking” via segmentation of the animalhttp://www.kinetrack.org2
Lekking
display
Stotting
behaviorSlide3
Goal
Automatic video object cut-outOffline analysis with minimum user inputBehavior analysis via post-processing
Input video
OutputSlide4
Previous Work
Bibby et al. ECCV’08
Bai
et al. SIGGRAPH’09
* Real-time, automatic
* Not focused on
segmentation
* High-quality segmentation
* Significant manual effort
is requiredSlide5
ApproachSegmentation in video volume MRF
Formulate as joint label assignmentNo shape priors, adaptation, etc.Joint label space encodes per-pixel segmentation and motionEnforce motion coherenceSegTrack database with video segmentation ground truthComparison to existing methods
5Slide6
p
MRF Multi-Label Space
t
+1
t
Temporal
neighbors
Spatial
neighbors
{ , }
X
{-2, -1, 0, +1, +2}Slide7
p
MRF Multi-Label Space
t
+1
t
Temporal
neighbors
Spatial
neighbors
{ , }
X
{-2, -1, 0, +1, +2}
{ , +1}Slide8
Find joint label assignment minimizing:
Multi-Label MRF Assignment
8
Data
SmoothnessSlide9
Data Term
9
t
+1
t
foreground
background
RGB likelihood
Optical flowSlide10
Smoothness Term
10
p
t
+1
t
Attribute
CoherenceSlide11
Smoothness Term
11
Attribute
Coherence
p
t
+1
t
Lowest
Cost
AssignmentSlide12
Smoothness Term
12
p
t
+1
t
Motion
CoherenceSlide13
Smoothness Term
13
p
t
+1
t
Motion
Coherence
Lowest
Cost
AssignmentSlide14
Optimization
Challenge: Combinatorics of label space162 labels/pixel => 19.4M labels/frameSolution:Process video in subvolumes, constrain the solution across the boundaries
Spatial
mult
-grid (factor of 16 savings)
Use
Fast-PD by Komodakis et.al. We have found Fast-PD to converge significantly faster than
graphcut20 seconds/frame for 300 x 400 imageSlide15
SegTrack Database
A new database for video segmentation with ground truthThree attributes that impact performance:Color overlap between target and backgroundInterframe motionChange in target shapeSlide16
Experiment
Quantitative Comparison with Chockalingam et.al ICCV’09Slide17
Quantitative Comparison
Comparison with Birchfield et.al ICCV 2009
Sequence
Color
Motion
Shape
Our Score
BirchfieldParachute
0.0380.1190.024
235
502
Girl
0.205
0.145
0.147
1304
1755
Monkeydog
0.299
0.243
0.132
563
683
Penguin
1.02
0.016
0.013
1705
6627
Birdfall
0.466
0.283
0.070
252
454
Cheetah
0.760
0.273
0.187
1142
1217
SegTrack Database metrics and scoresSlide18
Video Results
18Slide19
ConclusionNew approach to video object cut-out based on temporally-coherent MRF
New SegTrack dataset to facilitate quantitative evaluations of segmentation performancePromising experimental results, including comparison to two recent methods19