PPT-MapReduce

Simplified Data Processing on Large Clusters Jeff Dean Sanjay Ghemawat Google OSDI 2004 Slides based on those by authors and other online sources Motivation Large scale data processing. ：. R. eusing . results . of . mapreduce. . jobs. Jun Fan. Outline. Introduction to . MapReduce. MapReduce. and its implementations such as . Hadoop. are common in . Facebook. , Yahoo, and Google as well as smaller companies. : Simplified Data Processing on Large Clusters. Jeffrey Dean & . Sanjay . Ghemawat. Appeared in:. OSDI '04: Sixth Symposium on Operating System Design and Implementation, San Francisco, CA, December, 2004. . Tsung. -Wei Huang. and Martin D. F. Wong. Department of Electrical and Computer Engineering (ECE). University of Illinois at Urbana-Champaign (UIUC), IL, USA. 2015 ACM International Symposium on Physical Design (ISPD). Simplified Data Processing on Large . Clusters. by Jeffrey Dean and Sanjay . Ghemawa. Presented by Jon Logan. Outline. Problem Statement / Motivation. An Example Program. MapReduce. . vs. Hadoop. GFS / HDFS. Paradigms Activity at Data Intensive Workshop. . Shantenu Jha represented by . Geoffrey Fox. gcf@indiana.edu. . . http://www.infomall.org. . http://www.futuregrid.org. . . http://salsahpc.indiana.edu. Efficient and scalable architectures to perform pleasingly parallel, MapReduce and iterative data intensive computations on cloud environments. Thilina. . Gunarathne. (tgunarat@indiana.edu). Advisor : . Parallel Computing. MapReduce. Examples. Parallel Efficiency. Assignment. Parallel Computing. Parallel efficiency with . p. processors. Traditional parallel computing:. focus on compute intensive tasks. Efficient and scalable architectures to perform pleasingly parallel, MapReduce and iterative data intensive computations on cloud environments. Thilina. . Gunarathne. (tgunarat@indiana.edu). Advisor : . HPC(Exascale) . Systems. June 24 2012. HPC . 2012 . Cetraro. , Italy. Geoffrey Fox. gcf@indiana.edu. . Informatics, Computing and Physics. Indiana . University . Bloomington. Science Computing Environments. : Stopping Group Attacks by Spotting Lockstep Behavior in Social. . Networks (WWW2013). BEUTEL, ALEX, WANHONG XU, VENKATESAN GURUSWAMI, . CHRISTOPHER PALOW, AND CHRISTOS FALOUTSOS.. GROUP 20. Outline. ”. Cathy O’Neil & Rachel . Schutt. , 2013. R & Hadoop. Compute squares. 2. R. # create a list of 10 integers. ints. <- 1:10. # equivalent to . ints. <- c(1,2,3,4,5,6,7,8,9,10). # compute the squares. Jimmy Lin. The iSchool. University of Maryland. Monday, March 30, 2009. This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States. See http://creativecommons.org/licenses/by-nc-sa/3.0/us/ for details. Source. MapReduce. : Simplified Data Processing in Large Clusters. . Jefferey. Dean and Sanjay . Ghemawat. . OSDI 2004. Example Scenario. 3. Genome data from roughly . one million users. 125 MB of data per user. MapReduce. Architecture. MapReduce. Internals. MapReduce. Examples. JobTracker. Interface. MapReduce. : A Real World Analogy. Coins Deposit. ?. MapReduce. : A Real World Analogy. Coins Deposit. Coins Counting Machine.