/
Learning Invariances and Hierarchies Learning Invariances and Hierarchies

Learning Invariances and Hierarchies - PowerPoint Presentation

danika-pritchard
danika-pritchard . @danika-pritchard
Follow
428 views
Uploaded On 2017-05-02

Learning Invariances and Hierarchies - PPT Presentation

Pierre Baldi University of California Irvine Two Questions If we solve computer vision we have pretty much solved AI ANNs vs BNNs and Deep Learning If we solve computer vision ID: 543895

computer solve invariances deep solve computer deep invariances learning nns algorithms recognize target audition vision

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Learning Invariances and Hierarchies" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Learning Invariances and Hierarchies

Pierre BaldiUniversity of California, IrvineSlide2

Two Questions

“If we solve computer vision, we have pretty much solved AI.” A-NNs vs

B-NNs and Deep Learning.Slide3

If we solve computer vision…Slide4

If we solve computer vision…

If we solve computer audition,….Slide5

If we solve computer vision…

If we solve computer audition,….If we solve computer olfaction,…Slide6

If we solve computer vision…

If we solve computer audition,….If we solve computer olfaction,…If we solve computer vision, how can we build computers that can prove Fermat’s last theorem? Slide7

Invariances

Invariances in audition. We can recognize a tune invariantly with respect to: intensity, speed, tonality, harmonization, instrumentation, style, background.Invariances in olfaction. We can recognize an odor invariantly with respect to: concentrations, humidity, pressure, winds, mixtures, background.Slide8

Non-Invariances

Invariances evolution did not care about (although we are still evolving!...)We cannot recognize faces upside down.We cannot recognize tunes played in reverse.We cannot recognize stereoisomers as such. Enantiomers smell differently.Slide9

A-NNs vs B-NNsSlide10

Origin of Invariances

Weight sharing and translational invariance.Can we quantify approximate weight sharing?Can we use approximate weight sharing to improve performance?Some of the invariance comes

from the architecture.

Some may come from the

learning rules.

Slide11

Learning Invariances

E

Hebb

s

ymmetric connections

w

ij

=

w

ji

111

11-1

1-11

Acyclic orientation of the Hypercube O(H)

Isometry

Isometry

Hebb

Hebb

O(H)

H

I(O(H))

I(H)Slide12

Deep Learning ≈ Deep Targets

Training set: (

x

i

,y

i

) or

i

=1, . . ., m

?Slide13

Deep Target AlgorithmsSlide14

Deep Target AlgorithmsSlide15

Deep Target AlgorithmsSlide16

Deep Target AlgorithmsSlide17

Deep Target AlgorithmsSlide18

In spite of the vanishing gradient problem, (and the Newton problem) nothing seems to beat back-propagation.Is backpropagation biologically plausible?Slide19

Mathematics of Dropout (Cheap Approximation to Training Full Ensemble)Slide20

Two Questions

“If we solve computer vision, we have pretty much solved AI.” A-NNs vs

B-NNs and Deep Learning.