PPT-Noriko Tomuro 1 CSC 578 Neural Networks and Deep Learning

Fall 201819 10 Capsule Overview Introduction to Capsule Noriko Tomuro 2 A Capsule Network CapsNet is a new approach proposed by Geoffrey Hinton although his original idea dates back to 1990s. Information Processing & Artificial Intelligence. New-Generation Models & Methodology for Advancing . AI & SIP. Li Deng . Microsoft Research, Redmond, . USA. Tianjin University, July 4, 2013 (Day 3). Deep Learning @ . UvA. UVA Deep Learning COURSE - Efstratios Gavves & Max Welling. LEARNING WITH NEURAL NETWORKS . - . PAGE . 1. Machine Learning Paradigm for Neural Networks. The Backpropagation algorithm for learning with a neural network. Deep Neural Networks . Huan Sun. Dept. of Computer Science, UCSB. March 12. th. , 2012. Major Area Examination. Committee. Prof. . Xifeng. . Yan. Prof. . Linda . Petzold. Prof. . Ambuj. Singh. The Future of Real-Time Rendering?. 1. Deep Learning is Changing the Way We Do Graphics. [Chaitanya17]. [Dahm17]. [Laine17]. [Holden17]. [Karras17]. [Nalbach17]. Video. “. Audio-Driven Facial Animation by Joint End-to-End Learning of Pose and Emotion”. Deep . Learning. James K . Baker, Bhiksha Raj. , Rita Singh. Opportunities in Machine Learning. Great . advances are being made in machine learning. Artificial Intelligence. Machine. Learning. After decades of intermittent progress, some applications are beginning to demonstrate human-level performance!. Fall 2018/19. 2. . Backpropagation. (Some figures adapted from . NNDL book. ). 0. Some Terminologies of Neural Networks. Noriko Tomuro. 2. “. N-layer. neural network” – By naming convention, we do NOT include the input layer because it doesn’t have parameters.. Introduction 2. Mike . Mozer. Department of Computer Science and. Institute of Cognitive Science. University of Colorado at Boulder. Hinton’s Brief History of Machine Learning. What was hot in 1987?. Fall 2018/19. 7. Recurrent Neural Networks. (Some figures adapted from . NNDL book. ). Recurrent Neural Networks. Noriko Tomuro. 2. Recurrent Neural Networks (RNNs). RNN Training. Loss Minimization. Bidirectional RNNs. Fall 2018/19. 3. Improving Neural Networks. (Some figures adapted from . NNDL book. ). Various Approaches to . Improve Neural Networks. Noriko Tomuro. 2. Cost functions. Quadratic. Cross Entropy. Log likelihood. Fall 2018/19. 9. Hopfield Networks, Boltzmann Machines. . Unsupervised Neural Networks. Noriko Tomuro. 2. Hopfield Networks. Concepts. Boltzmann Machines. Concepts. Restricted Boltzmann Machines. Deep Boltzmann Machines. Weifeng Li, . Victor Benjamin, Xiao . Liu, and . Hsinchun . Chen. University of Arizona. 1. Acknowledgements. Many of the pictures, results, and other materials are taken from:. Aarti. Singh, Carnegie Mellon University. Secada combs | bus-550. AI Superpowers: china, silicon valley, and the new world order. Kai Fu Lee. Author of AI Superpowers. Currently Chairman and CEO of . Sinovation. Ventures and President of . Sinovation. Dr David Wong. (With thanks to Dr Gari Clifford, G.I.T). The Multi-Layer Perceptron. single layer can only deal with linearly separable data. Composed of many connected neurons . Three general layers; . Eli Gutin. MIT 15.S60. (adapted from 2016 course by Iain Dunning). Goals today. Go over basics of neural nets. Introduce . TensorFlow. Introduce . Deep Learning. Look at key applications. Practice coding in Python.