Pierre . Baldi. University of California, Irvine. Two Questions. “If we solve computer vision, we have pretty much solved AI.” . A-NNs . vs. B-NNs and Deep Learning.. If we solve computer vision…. ID: 543895 Download Presentation

Pierre . Baldi. University of California, Irvine. Two Questions. “If we solve computer vision, we have pretty much solved AI.” . A-NNs . vs. B-NNs and Deep Learning.. If we solve computer vision….

Tags :
computer solve
invariances deep
solve
computer
deep
invariances
learning
nns
algorithms
recognize
target
audition
vision

Download Presentation

Download Presentation - The PPT/PDF document "Learning Invariances and Hierarchies" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Learning Invariances and Hierarchies

Pierre BaldiUniversity of California, Irvine

Slide2Two Questions

“If we solve computer vision, we have pretty much solved AI.”

A-NNs

vs

B-NNs and Deep Learning.

Slide3If we solve computer vision…

Slide4If we solve computer vision…

If we solve computer audition,….

Slide5If we solve computer vision…

If we solve computer audition,….

If we solve computer olfaction,…

Slide6If we solve computer vision…

If we solve computer audition,….

If we solve computer olfaction,…

If we solve computer vision, how can we build computers that can prove Fermat’s last theorem?

Slide7Invariances

Invariances in audition. We can recognize a tune invariantly with respect to: intensity, speed, tonality, harmonization,

i

nstrumentation, style, background.

Invariances in olfaction. We can recognize an odor invariantly with respect to: concentrations, humidity, pressure, winds, mixtures, background.

Slide8Non-Invariances

Invariances evolution did not care about (although we are still evolving!...)

We cannot recognize faces upside down.

We cannot recognize tunes played in reverse.

We cannot recognize stereoisomers as such. Enantiomers smell differently.

Slide9A-NNs vs B-NNs

Slide10Origin of Invariances

Weight sharing and translational invariance.Can we quantify approximate weight sharing?Can we use approximate weight sharing to improve performance?Some of the invariance comes from the architecture. Some may come from the learning rules.

Slide11Learning Invariances

E

Hebb

s

ymmetric connections

w

ij

=

w

ji

111

11-1

1-11

Acyclic orientation of the Hypercube O(H)

Isometry

Isometry

Hebb

Hebb

O(H)

H

I(O(H))

I(H)

Slide12Deep Learning ≈ Deep Targets

Training set: (

x

i

,y

i

) or

i

=1, . . ., m

?

Slide13Deep Target Algorithms

Slide14Deep Target Algorithms

Slide15Deep Target Algorithms

Slide16Deep Target Algorithms

Slide17Deep Target Algorithms

Slide18

In spite of the vanishing gradient problem, (and the Newton problem) nothing seems to beat back-propagation.

Is

backpropagation

biologically plausible?

Slide19Mathematics of Dropout (Cheap Approximation to Training Full Ensemble)

Slide20Two Questions

“If we solve computer vision, we have pretty much solved AI.”

A-NNs

vs

B-NNs and Deep Learning.

© 2020 docslides.com Inc.

All rights reserved.