PPT-Distributed versus Sparse Representations

and Unsupervised Learning Chapter 5 Anastasio Learning objectives explain the difference between sparse and distributed neuronal encoding explain the difference in purpose between Hopfield networks and twolayer neural networks. Mountain View mikolovgooglecom Ilya Sutskever Google Inc Mountain View ilyasugooglecom Kai Chen Google Inc Mountain View kaigooglecom Greg Corrado Google Inc Mountain View gcorradogooglecom Jeffrey Dean Google Inc Mountain View jeffgooglecom Abstrac nyuedu Abstract We describe a novel unsupervised method for learning sparse overcomplete fea tures The model uses a linear encoder and a linear decoder p receded by a spar sifying nonlinearity that turns a code vector into a quasi binary sparse code Such matrices has several attractive properties they support algorithms with low computational complexity and make it easy to perform in cremental updates to signals We discuss applications to several areas including compressive sensing data stream Easy to understand Easy to code by hand Often used to represent inputs to a net Easy to learn This is what mixture models do Each cluster corresponds to one neuron Easy to associate with other representations or responses But localist models are ver Recovery. . (. Using . Sparse. . Matrices). Piotr. . Indyk. MIT. Heavy Hitters. Also called frequent elements and elephants. Define. HH. p. φ. . (. x. ) = { . i. : |x. i. | ≥ . φ. ||. x||. p. Natural Language Processing. Tomas Mikolov, Facebook. ML Prague 2016. Structure of this talk. Motivation. Word2vec. Architecture. Evaluation. Examples. Discussion. Motivation. Representation of text is very important for performance of many real-world applications: search, ads recommendation, ranking, spam filtering, …. Chao Xing. CSLT Tsinghua. Why?. Chris Dyer. group had gotten a lot of brilliant achievements in 2015, and their research interest match to ours. . And in some area, we two groups almost think same way, but we didn’t do so well as they did.. Gonzalo . Mateos. , Juan A. Bazerque, and . Georgios. B. Giannakis . Acknowledgement:. . NSF grants . CCF-0830480, 1016605 and . ECCS-0824007. January 6, 2011. Distributed sparse estimation. 2. Data acquired by . to Multiple Correspondence . Analysis. G. Saporta. 1. , . A. . . Bernard. 1,2. , . C. . . Guinot. 2,3. 1 . CNAM, Paris, France. 2 . CE.R.I.E.S., Neuilly sur Seine, France. 3 . Université. . François Rabelais. Michael . Elad. The Computer Science Department. The . Technion. – Israel Institute of technology. Haifa 32000, . Israel. David L. Donoho. Statistics Department Stanford USA. Author: . Vikas. . Sindhwani. and . Amol. . Ghoting. Presenter: . Jinze. Li. Problem Introduction. we are given a collection of N data points or signals in a high-dimensional space R. D. : xi ∈ . and their Compositionality. Presenter: Haotian Xu. Roadmap. Overview. The Skip-gram Model with Different . Objective Functions. Subsampling of Frequent Words. Learning Phrases. CNN for Text Classification. Parallelization of Sparse Coding & Dictionary Learning Univeristy of Colorado Denver Parallel Distributed System Fall 2016 Huynh Manh 11/15/2016 1 Contents Introduction to Sparse Coding Applications of Sparse Representation 2How can new representations be acquired When that question is asked about new concepts Fodor famously argued that hypothesis testing is the only option Fodor 1975 1981 That led him to embrace radica