PPT-From SqueezeNet to SqueezeBERT: Developing efficient deep neural networks

Developing efficient deep neural networks Forrest Iandola 1 Albert Shaw 2 Ravi Krishna 3 Kurt Keutzer 4 1 UC Berkeley DeepScale Tesla Independent Researcher 2 Georgia Tech DeepScale Tesla. Deep Learning @ . UvA. UVA Deep Learning COURSE - Efstratios Gavves & Max Welling. LEARNING WITH NEURAL NETWORKS . - . PAGE . 1. Machine Learning Paradigm for Neural Networks. The Backpropagation algorithm for learning with a neural network. Deep Neural Networks . Huan Sun. Dept. of Computer Science, UCSB. March 12. th. , 2012. Major Area Examination. Committee. Prof. . Xifeng. . Yan. Prof. . Linda . Petzold. Prof. . Ambuj. Singh. Deep . Learning. James K . Baker, Bhiksha Raj. , Rita Singh. Opportunities in Machine Learning. Great . advances are being made in machine learning. Artificial Intelligence. Machine. Learning. After decades of intermittent progress, some applications are beginning to demonstrate human-level performance!. Introduction 2. Mike . Mozer. Department of Computer Science and. Institute of Cognitive Science. University of Colorado at Boulder. Hinton’s Brief History of Machine Learning. What was hot in 1987?. Ali Cole. Charly. . Mccown. Madison . Kutchey. Xavier . henes. Definition. A directed network based on the structure of connections within an organism's brain. Many inputs and only a couple outputs. Fall 2018/19. 7. Recurrent Neural Networks. (Some figures adapted from . NNDL book. ). Recurrent Neural Networks. Noriko Tomuro. 2. Recurrent Neural Networks (RNNs). RNN Training. Loss Minimization. Bidirectional RNNs. Weifeng Li, . Victor Benjamin, Xiao . Liu, and . Hsinchun . Chen. University of Arizona. 1. Acknowledgements. Many of the pictures, results, and other materials are taken from:. Aarti. Singh, Carnegie Mellon University. Secada combs | bus-550. AI Superpowers: china, silicon valley, and the new world order. Kai Fu Lee. Author of AI Superpowers. Currently Chairman and CEO of . Sinovation. Ventures and President of . Sinovation. CS295: Modern Systems: Application Case Study Neural Network Accelerator – 2 Sang-Woo Jun Spring 2019 Many slides adapted from Hyoukjun Kwon‘s Gatech “Designing CNN Accelerators ” and Dr David Wong. (With thanks to Dr Gari Clifford, G.I.T). The Multi-Layer Perceptron. single layer can only deal with linearly separable data. Composed of many connected neurons . Three general layers; . Lingxiao Ma. . †. , Zhi Yang. . †. , Youshan Miao. ‡. , Jilong Xue. ‡. , Ming Wu. ‡. , Lidong Zhou. ‡. , . Yafei. Dai. . †. †. . Peking University. ‡ . Microsoft Research. USENIX ATC ’19, Renton, WA, USA. Short-Term . Memory. Recurrent . Neural Networks. Meysam. . Golmohammadi. meysam@temple.edu. Neural Engineering Data Consortium. College . of Engineering . Temple University . February . 2016. Introduction. Mark Hasegawa-Johnson. April 6, 2020. License: CC-BY 4.0. You may remix or redistribute if you cite the source.. Outline. Why use more than one layer?. Biological inspiration. Representational power: the XOR function. Eli Gutin. MIT 15.S60. (adapted from 2016 course by Iain Dunning). Goals today. Go over basics of neural nets. Introduce . TensorFlow. Introduce . Deep Learning. Look at key applications. Practice coding in Python.