Tucker Hermans James M Rehg Aaron Bobick Computational Perception Lab School of Interactive Computing Georgia Institute of Technology Motivation Determine applicable actions for an object of interest ID: 228477
Download Presentation The PPT/PDF document "Affordance Prediction via Learned Object..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Affordance Prediction via Learned Object Attributes
Tucker Hermans James M. Rehg Aaron BobickComputational Perception LabSchool of Interactive ComputingGeorgia Institute of TechnologySlide2
Motivation
Determine applicable actions for an object of interestLearn this ability for previously unseen objects
1Slide3
Affordances
Latent actions available in the environmentJoint function of the agent and objectProposed by Gibson 19772Slide4
Direct Perception
Affordances are directly perceived from the environmentGibson’s original model of affordance perceptionDirect Perception Model
3Slide5
Object Models
Category Affordance FullCategory Affordance Chain
4
Moore, Sun, Bobick, & Rehg, IJRR 2010Slide6
Attribute Affordance Model
Benefits of AttributesAttributes determine affordancesScale to novel object categoriesGive a supervisory signal not present in feature selectionAttribute-Affordance Model
5Slide7
Attribute Affordance Model
6Based on
Lampert et. al. CVPR 09Slide8
Visual Features
SIFT codewords extracted densely7
…
LAB color histogram
Texton
filter bankSlide9
Attributes
Shape: 2D-Boxy, 3D-Boxy, cylindrical, sphericalColors: blue, red, yellow, purple, green, orange, black, white, and grayMaterial: cloth, ceramic, metal, paper, plastic, rubber, and woodSize: height and width (cm)Weight (kg)Total attribute feature length: 23 total elements
8Slide10
Attribute Classifiers
Learn attribute classifiers using binary SVM and SVM regressionUse multichannel χ2 kernel9Slide11
Affordance Classifiers
Binary SVM with multichannel Euclidean and hamming distance kernelTrain on ground truth attribute valuesInfer affordance using predicted attribute values10Slide12
Experimental Setup
11Slide13
Experimental Data
Six object categories: balls, books, boxes, containers, shoes, and towels7 Affordances: rollable, pushable, gripable, liftable, traversable,
caryable, dragable375 total images
12Slide14
Results: Affordance Prediction
Attribute-Affordance13
Category Affordance ChainSlide15
Results: Affordance Prediction
Category Affordance Full14
Attribute-AffordanceSlide16
Results: Affordance Prediction
Category Affordance ChainCategory Affordance Full15Slide17
Results: Affordance Prediction
Attribute-AffordanceDirect Perception
16Slide18
Results: Affordance Prediction
AttributeDPCA-Full
CA-ChainPushable
74.43
83.75
77.50
65.56
Rollable
96.87
97.32
90.71
84.14
Graspable
70.09
81.25
73.21
55.48
Liftable
73.91
83.93
75.71
67.48
Dragable
72.87
81.43
75.00
60.00
Carryable
73.91
83.93
75.71
67.48
Traversable
93.39
95.00
90.71
86.61
Total
81.12
85.46
79.21
68.57
17
Percent correctly classifiedSlide19
Results: Attribute Prediction
Color PredictionMaterial Prediction
18Slide20
Results: Attribute Prediction
Shape PredictionObject Category Prediction
19Slide21
Results: Novel Object Class
Attribute-AffordanceDirect Perception20
Object class “book”Slide22
Results: Novel Object Class
Attribute-AffordanceDirect Perception21
Object class “box”Slide23
Results: Novel Object Class
BallsBooksBoxes
ContainerShoes
Towels
Attribute
52.03
39.99
69.01
76.28
60.97
53.63
DP
57.99
65.58
67.69
58.96
67.86
67.91
22
Percent of correctly classified affordances across all novel object categoriesSlide24
Future Work
Train attribute classifiers on larger auxiliary datasetIncorporate depth sensingCombine attribute and object modelsUse parts as well as attributesAffordances of elements other than individual objectsAttribute-Category Model
23Slide25
Conclusion
Current dataset does not provide a diverse enough set of object classes for attributes to provide significant information transferAttribute model restricts use of all features, unlike direct perception which has all visual features availableAttribute model outperformed object modelsDirect perception and attribute models are comparable for small amounts of training data24Slide26
Affordance Prediction via Learned Object Attributes
Tucker Hermans James M. Rehg Aaron BobickComputational Perception LabSchool of Interactive ComputingGeorgia Institute of Technology