/
The memorability of visual images The memorability of visual images

The memorability of visual images - PowerPoint Presentation

phoebe-click
phoebe-click . @phoebe-click
Follow
344 views
Uploaded On 2019-02-09

The memorability of visual images - PPT Presentation

Khosla Raju Torralba Oliva 2015 h igh memorability low memorability Amazon Mechanical Turk o bjective memorability scores 60000 photographs consistency across observers LaMem ID: 751257

memorable memorability memnet image memorability memorable image memnet strong cnn regions oliva torralba visual emorability cortex map predicting hippocampus

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "The memorability of visual images" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

The memorability of visual imagesKhosla, Raju, Torralba, Oliva (2015)

h

igh memorability

low memorability

Amazon Mechanical Turk:

objective memorability scores 60,000 photographsconsistency across observers

LaMem databaseSlide2

Most memorable

Least memorable

Focus

Enclosed Setting

DynamicsUnusual

No single focus

Distant

view

Static

Common

You need to recognize to remember!Slide3

Scene Understanding:

Context and ObjectsSlide4

first t

rained CNN with places +

imageNet image databases

AlexNet

Aditya

KhoslaAgataLapedriza

BoleiZhou

Antonio

Torralba

10 million images, 460 categoriesSlide5

input

output

LaMem

m

emorability score

Khosla, Raju,

Torralba

,

Oliva (2016)

Understanding and Predicting Image

Memorability at a Large Scale

. http://

memorability.csail.mit.edu

/

MemNet

CNN for predicting

i

mage

m

emorabilitySlide6

Visualizing CNN “features”

a

verage of images that maximally activate neurons in two layers near output of

MemNet

(ordered (top to bottom) by correlation with memorability) Slide7

strong positive

strong negative

Visualizing

receptive field

of

MemNet neurons with strong positive or negative correlation with memorabilitySlide8

Saliency: which image regions are memorable?

g

enerate a

memorability map:

scale up the image

size, apply MemNet to overlapping subregions

use memorability map to de-emphasize the less memorable details

AMT experiment shows highlighted regions contribute to higher memorability scoresSlide9

Preserving most memorable regions

a

pplications of memorability work?Slide10

PPA

RSC

OPA

LOC

Neural framework of memorability

Perception

(ventral/dorsal visual cortex)Memory

(Medial Temporal Lobe)

a

ttention network

e

arly

visual areas

Wilma

BainbridgeSlide11

Whole Brain Contrast

m

emorable

forgettable

novel

novelSlide12

Multi-variate Pattern Analysis

Memorable vs. Forgettable

Hippocampus (Posterior)

MVPA Searchlight preliminary results (3x3x3 cubic voxel grid)

role for the hippocampus in higher-order statistical perception

PerirhinalParahippocampal Cortex

Bainbridge, Dilks

, Oliva (in preparation)