PPT-Map reduce 5/24/2011 Map Reduce

Author : celsa-spraggs | Published Date : 2019-06-26

1 Word Count Example We have a large file of words one word to a line Count the number of times each distinct word appears in the file Sample application analyze

Presentation Embed Code

Download Presentation

Download Presentation The PPT/PDF document "Map reduce 5/24/2011 Map Reduce" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Map reduce 5/24/2011 Map Reduce: Transcript

1 Word Count Example We have a large file of words one word to a line Count the number of times each distinct word appears in the file Sample application analyze web server logs to find popular URLs. S. A. L. S. A. . HPC Group . http://. salsahpc.indiana.edu. School of Informatics and Computing. Indiana University. Judy . Qiu. . Thilina. . Gunarathne. . CAREER Award. Outline. Iterative . Lecture 9. Outline. Map Reduce, cont.. Index compression. [Amazon Web Services]. Map Reduce. Map: (key, value) . . . list(key. ’, value’). Reduce: (key’, . list(value. ’) . . list (value’). Data Analysis on . MapReduce. Chao Liu, Hung-. chih. Yang, Jinliang Fan, Li-Wei He, Yi-Min Wang. Internet Services Research Center (ISRC). Microsoft Research Redmond. Internet Services Research Center (ISRC). Nima Sarshar, Ph.D.. INTUIT . Inc. ,. Nima_sarshar@intuit.com . Intuit, . Graphs and Me. Me: . Large-scale graph data processing, complex networks analysis, graph algorithms … . Intuit: . QuickBooks, TurboTax, . Origins & Applications. Christopher Smith. Xavier Stevens. John Carnahan. Original Map & Reduce. LISP. map f(x) [x. 0. , x. 1. , …, x. n. ]. yields: [f(x. 0. ), f(x. 1. ), …, f(x. n. )]. reduce f(x, y) [x. MapReduce. Outline. MapReduce. Architecture. MapReduce. Internals. MapReduce. Examples. JobTracker. Interface. MapReduce. : A Real World Analogy. Coins Deposit. ?. MapReduce. : A Real World Analogy. MapReduce. Based on slides from Jimmy Lin’s lecture slides (http://www.umiacs.umd.edu/~jimmylin/cloud-2010-Spring/index.html) (licensed under Creation Commons Attribution 3.0 License). Slides from . What-if Analysis, and . Cost-based Optimization of . MapReduce. Programs. Herodotos Herodotou. Shivnath Babu. Duke University. Analysis in the Big Data Era. 8/31/2011. Duke University. 2. Popular option. Science Applications. http://salsahpc.indiana.edu . School of Informatics and Computing. Indiana University. Application Classes. 1. Synchronous. Lockstep Operation as in SIMD architectures. SIMD. 2. Large Scale Duplicate Detection. Prof. Felix . Naumann. , . Arvid. . Heise. Agenda. Big Data. Word Count Example. Hadoop. Distributed File System. Hadoop. . Map/Reduce. Advanced . Map/Reduce. Stratosphere. Yang PENG . Network and System Lab. CSE, HKUST. Monday, March 11, 2013. ypengab@cse.ust.hk. Material adapted from slides by Christophe . Bisciglia. , Aaron Kimball, & Sierra . Michels-Slettvet. Condie. UC . Berkeley. Slides by . Kaixiang. MO. kxmo@cse.ust.hk. Outline. Background. Motivation: Block . vs. . pipeline. Hadoop. Online Prototype Model. Pipeline within job. Online Aggregation. Continuous Queries. Madhu M Nayak Assistant Professor, Department of C SE, GSSSIETW, Mysuru Pradeep.S Assistant Professor, Department of C SE, GEC, Kushal Nagar Abstract - Due to the increasing popularity of cheap Jiaul. Paik. Email:. . jia.paik@gmail.com. Today’s Topics. Map-reduce: Additional Details. Inverted Index using Map-reduce. Introduction to Spark. Spark Demo. Map-reduce Internals: Additional Details.

Download Document

Here is the link to download the presentation.
"Map reduce 5/24/2011 Map Reduce"The content belongs to its owner. You may download and print it for personal use, without modification, and keep all copyright notices. By downloading, you agree to these terms.