PPT-Training DNNs with up to 5x less memory via optimal tensor

Author : SunnySeahorse | Published Date : 2022-08-01

rematerialization Paras Jain Joint work with Ajay Jain Ani Nrusimha Amir Gholami Pieter Abbeel Kurt Keutzer Ion Stoica Joseph Gonzalez To appear at MLSys

Presentation Embed Code

Download Presentation

Download Presentation The PPT/PDF document "Training DNNs with up to 5x less memory ..." is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Training DNNs with up to 5x less memory via optimal tensor: Transcript

rematerialization Paras Jain Joint work with Ajay Jain Ani Nrusimha Amir Gholami Pieter Abbeel Kurt Keutzer Ion Stoica Joseph Gonzalez To appear at MLSys 2020 Deep learning continues to adapt to increasing complex applications. Avg Access Time 2 Tokens Number of Controllers Average Access Time clock cyles brPage 16br Number of Tokens vs Avg Access Time 9 Controllers Number of Tokens Average Access Time clock cycles brPage 17br brPage 18br IndexTopicarraytensor,1%*t%(tensor),1%t*%(tensor),1%t*t%(tensor),1aperm,2matmult,2tensor,14 He Zhang. 1. , He Huang. 2. , . Rui. Li. 1. , . Jie. Chen. 1. , Li-Shi Luo. 2. Jefferson Lab. Old Dominion University. FEIS-2, 05/15/2015. Outline. He Zhang. ---. 3. ---. Introduction of FMM. He Zhang. Overview. Theory. Basic . physics. Tensor. Diffusion . imaging . Practice. How . do you do DTI?. . Tractography. . DTI . in . FSL and other programs. Diffusion . Tensor Imaging. Brownian motion. Overview. Theory. Basic . physics. Tensor. Diffusion . imaging . Practice. How . do you do DTI?. . Tractography. . DTI . in . FSL and other programs. Diffusion . Tensor Imaging. Brownian motion. Shenghan Jiang. Boston College. Benasque. February. , 09, 2017. Symmetric tensor-networks and topological phases. Collaborators:. Ying Ran (Boston College) . Panjin. Kim, . Hyungyong. Lee, Jung . Hoon. He Zhang. 1. , He Huang. 2. , . Rui. Li. 1. , . Jie. Chen. 1. , Li-Shi Luo. 2. Jefferson Lab. Old Dominion University. FEIS-2, 05/15/2015. Outline. He Zhang. ---. 3. ---. Introduction of FMM. He Zhang. Presenters: Vicente Ordonez, Paola Cascante. Motivation. Two grand challenges in AI: . Models that can make multiple computational steps to answer a question or completing a task. Models that can describe long term dependencies in sequential data. Juan Andrés . Bazerque. , Gonzalo . Mateos. , and . Georgios. B. . Giannakis. . August. 8, 2012. . Spincom. group, University of Minnesota. . Acknowledgment: . AFOSR MURI grant no. FA 9550-10-1-0567. Elena Long. High Energy Nuclear Physics With . Spectator Tagging, ODU. March 11. th. , 2015. 1. 03/11/2015. Spectator Tagging Workshop . . Elena Long <ellie@jlab.org>. Today’s Discussion. Background. T. Chen, T. Moreau, Z. Jiang, L. Zheng, S. Jiao, E. Yan, H. Shen, M. Cowan, L. Wang, Y. Hu, L. . Ceze. , C. . Guestrin. , and A. Krishnamurthy . Presentation by Grzegorz . Topics: . Diffy. , Morph, Gradient Compression. 3D CNNs. Used for video processing. Examining a series of F images in one step. T is typically 3. Note that F reduces as we advance (also because of pooling). THE GENESIS OF SEISMIC EVENTS IN THE EAST BALTIC REGION. D.Malytskyy. * & V.Nikulins**. . * . Carpat. h. ian. . Branch of . Subbotin. Institute of Geophysics, National Academy of Sciences of Ukraine, Ukraine. Jing Chen, PhD . Department of Psychology. Grand Valley State University. Chris . Kurby. , PhD. Department of Psychology. Grand Valley State University. The 11. th. . Annual Art and Science of Aging Conference.

Download Document

Here is the link to download the presentation.
"Training DNNs with up to 5x less memory via optimal tensor"The content belongs to its owner. You may download and print it for personal use, without modification, and keep all copyright notices. By downloading, you agree to these terms.