PPT-Introduction to map-reduce

Jiaul Paik Email jiapaikgmailcom Todays Topics Mapreduce Additional Details Inverted Index using Mapreduce Introduction to Spark Spark Demo Mapreduce Internals Additional Details. Lecture 9. Outline. Map Reduce, cont.. Index compression. [Amazon Web Services]. Map Reduce. Map: (key, value) . . . list(key. ’, value’). Reduce: (key’, . list(value. ’) . . list (value’). : Distributed Co-clustering with Map-Reduce. S. Papadimitriou, J. Sun. IBM T.J. Watson Research Center. Speaker:. 0356169. 吳宏君. 0350741. . 陳威遠. 0356042 . 洪浩哲. Outline. Introduction. Origins & Applications. Christopher Smith. Xavier Stevens. John Carnahan. Original Map & Reduce. LISP. map f(x) [x. 0. , x. 1. , …, x. n. ]. yields: [f(x. 0. ), f(x. 1. ), …, f(x. n. )]. reduce f(x, y) [x. Some . Advanced Topics. 15-213 / 18-213: Introduction to Computer Systems. 27. th. Lecture, Dec.. 4, 2012. Instructors:. . Dave O’Hallaron, Greg Ganger, and Greg . Kesden. Today. Library . interpositioning. MapReduce. By Tomasz . Nykiel. . (University of Toronto). Michalis. . Potamias. (Boston University). Chaitanya. . Mishra. (University of Toronto, currently . Facebook. ). George . Kollios. (Boston University). Science Applications. http://salsahpc.indiana.edu . School of Informatics and Computing. Indiana University. Application Classes. 1. Synchronous. Lockstep Operation as in SIMD architectures. SIMD. 2. Thilina Gunarathne (. tgunarat@indiana.edu. ). Judy . Qiu (. xqiu@indiana.edu. ). Dennis . Gannon (. dennis.gannon@microsoft.com). Introduction. Three disruptions. Big Data. MapReduce. Cloud Computing. Mega 10^6. Giga 10^9. Tera. 10^12. Peta. 10^15. Pig Latin. Understanding . MapReduce. . in Context. Linux HPC. Bare-system. Amazon Cloud. Windows Server HPC. Bare-system. Virtualization. Cross Platform Iterative . Thilina Gunarathne (. tgunarat@indiana.edu. ). Judy . Qiu (. xqiu@indiana.edu. ). Dennis . Gannon (. dennis.gannon@microsoft.com). Introduction. Three disruptions. Big Data. MapReduce. Cloud Computing. : Simplified Data Processing on Large Clusters . Jeff Dean, Sanjay . Ghemawat. , Google, OSDI 2004. Slides based on those by . authors and other online sources. Motivation. Large . scale data processing. Yang PENG . Network and System Lab. CSE, HKUST. Monday, March 11, 2013. ypengab@cse.ust.hk. Material adapted from slides by Christophe . Bisciglia. , Aaron Kimball, & Sierra . Michels-Slettvet. Basics. Divide and conquer. Partition large problem into smaller . subproblems. Worker work on . subproblems. in parallel. Threads in a core, cores in multi-core processor, multiple processor in a machine, machines in a cluster. Madhu M Nayak Assistant Professor, Department of C SE, GSSSIETW, Mysuru Pradeep.S Assistant Professor, Department of C SE, GEC, Kushal Nagar Abstract - Due to the increasing popularity of cheap Cast of thousands. Mihai. Pop. Michael Schatz. Dan . Sommer. University of Maryland Center for Computational Biology. Faisal Khan, Ken Hahn UW . David Schwartz, LMCG. In 2003…. http://labs.google.com/papers/gfs.html.