Khosla Raju Torralba Oliva 2015 h igh memorability low memorability Amazon Mechanical Turk o bjective memorability scores 60000 photographs consistency across observers LaMem ID: 751257
Download Presentation The PPT/PDF document "The memorability of visual images" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
The memorability of visual imagesKhosla, Raju, Torralba, Oliva (2015)
h
igh memorability
low memorability
Amazon Mechanical Turk:
objective memorability scores 60,000 photographsconsistency across observers
LaMem databaseSlide2
Most memorable
Least memorable
Focus
Enclosed Setting
DynamicsUnusual
No single focus
Distant
view
Static
Common
You need to recognize to remember!Slide3
Scene Understanding:
Context and ObjectsSlide4
first t
rained CNN with places +
imageNet image databases
AlexNet
Aditya
KhoslaAgataLapedriza
BoleiZhou
Antonio
Torralba
10 million images, 460 categoriesSlide5
input
output
LaMem
m
emorability score
Khosla, Raju,
Torralba
,
Oliva (2016)
Understanding and Predicting Image
Memorability at a Large Scale
. http://
memorability.csail.mit.edu
/
MemNet
CNN for predicting
i
mage
m
emorabilitySlide6
Visualizing CNN “features”
a
verage of images that maximally activate neurons in two layers near output of
MemNet
(ordered (top to bottom) by correlation with memorability) Slide7
strong positive
strong negative
Visualizing
receptive field
of
MemNet neurons with strong positive or negative correlation with memorabilitySlide8
Saliency: which image regions are memorable?
g
enerate a
memorability map:
scale up the image
size, apply MemNet to overlapping subregions
use memorability map to de-emphasize the less memorable details
AMT experiment shows highlighted regions contribute to higher memorability scoresSlide9
Preserving most memorable regions
a
pplications of memorability work?Slide10
PPA
RSC
OPA
LOC
Neural framework of memorability
Perception
(ventral/dorsal visual cortex)Memory
(Medial Temporal Lobe)
a
ttention network
e
arly
visual areas
Wilma
BainbridgeSlide11
Whole Brain Contrast
m
emorable
forgettable
novel
novelSlide12
Multi-variate Pattern Analysis
Memorable vs. Forgettable
Hippocampus (Posterior)
MVPA Searchlight preliminary results (3x3x3 cubic voxel grid)
role for the hippocampus in higher-order statistical perception
PerirhinalParahippocampal Cortex
Bainbridge, Dilks
, Oliva (in preparation)