Pierre Baldi University of California Irvine Two Questions If we solve computer vision we have pretty much solved AI ANNs vs BNNs and Deep Learning If we solve computer vision ID: 543895
Download Presentation The PPT/PDF document "Learning Invariances and Hierarchies" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Learning Invariances and Hierarchies
Pierre BaldiUniversity of California, IrvineSlide2
Two Questions
“If we solve computer vision, we have pretty much solved AI.” A-NNs vs
B-NNs and Deep Learning.Slide3
If we solve computer vision…Slide4
If we solve computer vision…
If we solve computer audition,….Slide5
If we solve computer vision…
If we solve computer audition,….If we solve computer olfaction,…Slide6
If we solve computer vision…
If we solve computer audition,….If we solve computer olfaction,…If we solve computer vision, how can we build computers that can prove Fermat’s last theorem? Slide7
Invariances
Invariances in audition. We can recognize a tune invariantly with respect to: intensity, speed, tonality, harmonization, instrumentation, style, background.Invariances in olfaction. We can recognize an odor invariantly with respect to: concentrations, humidity, pressure, winds, mixtures, background.Slide8
Non-Invariances
Invariances evolution did not care about (although we are still evolving!...)We cannot recognize faces upside down.We cannot recognize tunes played in reverse.We cannot recognize stereoisomers as such. Enantiomers smell differently.Slide9
A-NNs vs B-NNsSlide10
Origin of Invariances
Weight sharing and translational invariance.Can we quantify approximate weight sharing?Can we use approximate weight sharing to improve performance?Some of the invariance comes
from the architecture.
Some may come from the
learning rules.
Slide11
Learning Invariances
E
Hebb
s
ymmetric connections
w
ij
=
w
ji
111
11-1
1-11
Acyclic orientation of the Hypercube O(H)
Isometry
Isometry
Hebb
Hebb
O(H)
H
I(O(H))
I(H)Slide12
Deep Learning ≈ Deep Targets
Training set: (
x
i
,y
i
) or
i
=1, . . ., m
?Slide13
Deep Target AlgorithmsSlide14
Deep Target AlgorithmsSlide15
Deep Target AlgorithmsSlide16
Deep Target AlgorithmsSlide17
Deep Target AlgorithmsSlide18
In spite of the vanishing gradient problem, (and the Newton problem) nothing seems to beat back-propagation.Is backpropagation biologically plausible?Slide19
Mathematics of Dropout (Cheap Approximation to Training Full Ensemble)Slide20
Two Questions
“If we solve computer vision, we have pretty much solved AI.” A-NNs vs
B-NNs and Deep Learning.