PPT-Map/Reduce

Large Scale Duplicate Detection Prof Felix Naumann Arvid Heise Agenda Big Data Word Count Example Hadoop Distributed File System Hadoop MapReduce Advanced MapReduce Stratosphere. : Simplified Data Processing on Large Clusters. Jeffrey Dean & . Sanjay . Ghemawat. Appeared in:. OSDI '04: Sixth Symposium on Operating System Design and Implementation, San Francisco, CA, December, 2004. . R. e. d. u. c. e. . Simplified Data Processing on Large Clusters. (Without the Agonizing Pain). Presented by Aaron Nathan. The Problem. Massive amounts of data. >100TB (the internet). Needs simple processing. S. A. L. S. A. . HPC Group . http://. salsahpc.indiana.edu. School of Informatics and Computing. Indiana University. Judy . Qiu. . Thilina. . Gunarathne. . CAREER Award. Outline. Iterative . 1. Abstractions On Top Of . Hadoop. We’ve decomposed some algorithms into a map-reduce “workflow” (series of map-reduce steps). naive Bayes training. naïve Bayes testing. phrase scoring. How else can we express these sorts of computations? Are there some common special cases of map-reduce steps we can parameterize and reuse?. : Distributed Co-clustering with Map-Reduce. S. Papadimitriou, J. Sun. IBM T.J. Watson Research Center. Speaker:. 0356169. 吳宏君. 0350741. . 陳威遠. 0356042 . 洪浩哲. Outline. Introduction. Data Analysis on . MapReduce. Chao Liu, Hung-. chih. Yang, Jinliang Fan, Li-Wei He, Yi-Min Wang. Internet Services Research Center (ISRC). Microsoft Research Redmond. Internet Services Research Center (ISRC). Nima Sarshar, Ph.D.. INTUIT . Inc. ,. Nima_sarshar@intuit.com . Intuit, . Graphs and Me. Me: . Large-scale graph data processing, complex networks analysis, graph algorithms … . Intuit: . QuickBooks, TurboTax, . Some . Advanced Topics. 15-213 / 18-213: Introduction to Computer Systems. 27. th. Lecture, Dec.. 4, 2012. Instructors:. . Dave O’Hallaron, Greg Ganger, and Greg . Kesden. Today. Library . interpositioning. MapReduce. Based on slides from Jimmy Lin’s lecture slides (http://www.umiacs.umd.edu/~jimmylin/cloud-2010-Spring/index.html) (licensed under Creation Commons Attribution 3.0 License). Slides from . Thilina Gunarathne (. tgunarat@indiana.edu. ). Judy . Qiu (. xqiu@indiana.edu. ). Dennis . Gannon (. dennis.gannon@microsoft.com). Introduction. Three disruptions. Big Data. MapReduce. Cloud Computing. : Simplified Data Processing on Large Clusters . Jeff Dean, Sanjay . Ghemawat. , Google, OSDI 2004. Slides based on those by . authors and other online sources. Motivation. Large . scale data processing. Yang PENG . Network and System Lab. CSE, HKUST. Monday, March 11, 2013. ypengab@cse.ust.hk. Material adapted from slides by Christophe . Bisciglia. , Aaron Kimball, & Sierra . Michels-Slettvet. Basics. Divide and conquer. Partition large problem into smaller . subproblems. Worker work on . subproblems. in parallel. Threads in a core, cores in multi-core processor, multiple processor in a machine, machines in a cluster. Madhu M Nayak Assistant Professor, Department of C SE, GSSSIETW, Mysuru Pradeep.S Assistant Professor, Department of C SE, GEC, Kushal Nagar Abstract - Due to the increasing popularity of cheap