Learning Invariances and Hierarchies

Learning Invariances and Hierarchies - Description

Pierre . Baldi. University of California, Irvine. Two Questions. “If we solve computer vision, we have pretty much solved AI.” . A-NNs . vs. B-NNs and Deep Learning.. If we solve computer vision…. ID: 543895 Download Presentation

50K - views

Learning Invariances and Hierarchies

Pierre . Baldi. University of California, Irvine. Two Questions. “If we solve computer vision, we have pretty much solved AI.” . A-NNs . vs. B-NNs and Deep Learning.. If we solve computer vision….

Similar presentations


Download Presentation

Learning Invariances and Hierarchies




Download Presentation - The PPT/PDF document "Learning Invariances and Hierarchies" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.



Presentation on theme: "Learning Invariances and Hierarchies"— Presentation transcript:

Slide1

Learning Invariances and Hierarchies

Pierre BaldiUniversity of California, Irvine

Slide2

Two Questions

“If we solve computer vision, we have pretty much solved AI.”

A-NNs

vs

B-NNs and Deep Learning.

Slide3

If we solve computer vision…

Slide4

If we solve computer vision…

If we solve computer audition,….

Slide5

If we solve computer vision…

If we solve computer audition,….

If we solve computer olfaction,…

Slide6

If we solve computer vision…

If we solve computer audition,….

If we solve computer olfaction,…

If we solve computer vision, how can we build computers that can prove Fermat’s last theorem?

Slide7

Invariances

Invariances in audition. We can recognize a tune invariantly with respect to: intensity, speed, tonality, harmonization,

i

nstrumentation, style, background.

Invariances in olfaction. We can recognize an odor invariantly with respect to: concentrations, humidity, pressure, winds, mixtures, background.

Slide8

Non-Invariances

Invariances evolution did not care about (although we are still evolving!...)

We cannot recognize faces upside down.

We cannot recognize tunes played in reverse.

We cannot recognize stereoisomers as such. Enantiomers smell differently.

Slide9

A-NNs vs B-NNs

Slide10

Origin of Invariances

Weight sharing and translational invariance.Can we quantify approximate weight sharing?Can we use approximate weight sharing to improve performance?Some of the invariance comes from the architecture. Some may come from the learning rules.

Slide11

Learning Invariances

E

Hebb

s

ymmetric connections

w

ij

=

w

ji

111

11-1

1-11

Acyclic orientation of the Hypercube O(H)

Isometry

Isometry

Hebb

Hebb

O(H)

H

I(O(H))

I(H)

Slide12

Deep Learning ≈ Deep Targets

Training set: (

x

i

,y

i

) or

i

=1, . . ., m

?

Slide13

Deep Target Algorithms

Slide14

Deep Target Algorithms

Slide15

Deep Target Algorithms

Slide16

Deep Target Algorithms

Slide17

Deep Target Algorithms

Slide18

In spite of the vanishing gradient problem, (and the Newton problem) nothing seems to beat back-propagation.

Is

backpropagation

biologically plausible?

Slide19

Mathematics of Dropout (Cheap Approximation to Training Full Ensemble)

Slide20

Two Questions

“If we solve computer vision, we have pretty much solved AI.”

A-NNs

vs

B-NNs and Deep Learning.