PPT-Other Map-Reduce (

ish Frameworks William Cohen 1 Outline More concise languages for mapreduce pipelines Abstractions built on top of mapreduce General comments Specific systems Cascading Pipes PIG Hive. : Simplified Data Processing on Large Clusters. Jeffrey Dean & . Sanjay . Ghemawat. Appeared in:. OSDI '04: Sixth Symposium on Operating System Design and Implementation, San Francisco, CA, December, 2004. . 1. Abstractions On Top Of . Hadoop. We’ve decomposed some algorithms into a map-reduce “workflow” (series of map-reduce steps). naive Bayes training. naïve Bayes testing. phrase scoring. How else can we express these sorts of computations? Are there some common special cases of map-reduce steps we can parameterize and reuse?. : Distributed Co-clustering with Map-Reduce. S. Papadimitriou, J. Sun. IBM T.J. Watson Research Center. Speaker:. 0356169. 吳宏君. 0350741. . 陳威遠. 0356042 . 洪浩哲. Outline. Introduction. Data Analysis on . MapReduce. Chao Liu, Hung-. chih. Yang, Jinliang Fan, Li-Wei He, Yi-Min Wang. Internet Services Research Center (ISRC). Microsoft Research Redmond. Internet Services Research Center (ISRC). Nima Sarshar, Ph.D.. INTUIT . Inc. ,. Nima_sarshar@intuit.com . Intuit, . Graphs and Me. Me: . Large-scale graph data processing, complex networks analysis, graph algorithms … . Intuit: . QuickBooks, TurboTax, . Origins & Applications. Christopher Smith. Xavier Stevens. John Carnahan. Original Map & Reduce. LISP. map f(x) [x. 0. , x. 1. , …, x. n. ]. yields: [f(x. 0. ), f(x. 1. ), …, f(x. n. )]. reduce f(x, y) [x. Steven Y. Ko. ,. Imranul. . Hoque. ,. Brian Cho, . and . Indranil. Gupta. Distributed Protocols Research Group (DPRG). University of Illinois at Urbana-Champaign. Our Position. Intermediate data as a first-class citizen . “MapReduce and parallel DBMSs: friends or foes?” Stonebraker, Daniel Abadi, David J Dewitt et al.. MAP REDUCE AND PARALLEL DBMS ARE COMPLEMENTARY. In 2010, . MapReduce. (MR) has been hailed as a . Science Applications. http://salsahpc.indiana.edu . School of Informatics and Computing. Indiana University. Application Classes. 1. Synchronous. Lockstep Operation as in SIMD architectures. SIMD. 2. Large Scale Duplicate Detection. Prof. Felix . Naumann. , . Arvid. . Heise. Agenda. Big Data. Word Count Example. Hadoop. Distributed File System. Hadoop. . Map/Reduce. Advanced . Map/Reduce. Stratosphere. : Simplified Data Processing on Large Clusters . Jeff Dean, Sanjay . Ghemawat. , Google, OSDI 2004. Slides based on those by . authors and other online sources. Motivation. Large . scale data processing. Yang PENG . Network and System Lab. CSE, HKUST. Monday, March 11, 2013. ypengab@cse.ust.hk. Material adapted from slides by Christophe . Bisciglia. , Aaron Kimball, & Sierra . Michels-Slettvet. Condie. UC . Berkeley. Slides by . Kaixiang. MO. kxmo@cse.ust.hk. Outline. Background. Motivation: Block . vs. . pipeline. Hadoop. Online Prototype Model. Pipeline within job. Online Aggregation. Continuous Queries. Cast of thousands. Mihai. Pop. Michael Schatz. Dan . Sommer. University of Maryland Center for Computational Biology. Faisal Khan, Ken Hahn UW . David Schwartz, LMCG. In 2003…. http://labs.google.com/papers/gfs.html.