PPT-Map-Reduce Abstractions

1 Abstractions On Top Of Hadoop Weve decomposed some algorithms into a mapreduce workflow series of mapreduce steps naive Bayes training naïve Bayes testing phrase scoring How else can we express these sorts of computations Are there some common special cases of mapreduce steps we can parameterize and reuse. Dictionary ADT. : Arrays, Lists and . Trees. Kate Deibel. Summer 2012. June 27, 2012. CSE 332 Data Abstractions, Summer 2012. 1. Where We Are. Studying the absolutely essential ADTs of computer science and classic data structures for implementing them. Chris Rossbach, Microsoft Research. Jon Currey, Microsoft Research. Emmett . Witchel. , University of Texas at Austin. HotOS. 2011. Lots of GPUs. Must they be so hard to use?. We need dataflow…. GPU Haiku . Nima Sarshar, Ph.D.. INTUIT . Inc. ,. Nima_sarshar@intuit.com . Intuit, . Graphs and Me. Me: . Large-scale graph data processing, complex networks analysis, graph algorithms … . Intuit: . QuickBooks, TurboTax, . Lecture 6: Programming languages. USABILITY OF NOTATIONS. Cognitive Dimensions of Notations. ‘Discussion tools’ for use when considering alternative designs of programming languages, and other complex information systems. String . Verification. Given a string manipulating program, . string . analysis . determines . all possible values . that a string expression can take during any program execution. Using string analysis we . Watercolor Paint - Abstractions. Watercolor Paint – Non-Representational. Pastel Abstractions. Large Scale Duplicate Detection. Prof. Felix . Naumann. , . Arvid. . Heise. Agenda. Big Data. Word Count Example. Hadoop. Distributed File System. Hadoop. . Map/Reduce. Advanced . Map/Reduce. Stratosphere. 6/16/2010. Parallel Programming Abstractions. 1. Tasks . vs. Threads. Similar but not the same.. 6/16/2010. Parallel Programming Abstractions. 2. h/w processors. Operating System. T. hreads. Task Scheduler. Eurocrypt. May 1. st. , 2017. Rafael Pass, Elaine Shi, . Florian Tramèr. Trusted hardware: . Different . communities. , different . world views. Crypto. Architecture. Systems. & . Security. “Minimal” trusted . 1. Recap. Map-reduce ✔️. Algorithms with multiple map-reduce steps. Naïve . bayes. test routine for large datasets and large models. Cleanly describing these algorithms. workflow (or dataflow) languages . Basics. Divide and conquer. Partition large problem into smaller . subproblems. Worker work on . subproblems. in parallel. Threads in a core, cores in multi-core processor, multiple processor in a machine, machines in a cluster. CSE 332 Data Abstractions: A Heterozygous Forest of AVL, Splay, and B Trees Kate Deibel Summer 2012 July 2, 2012 CSE 332 Data Abstractions, Summer 2012 1 From last time… Binary search trees can give us great performance due to providing a structured binary search. Lecture 19: Analysis of Fork-Join Parallel Programs. Dan Grossman. Spring 2010. Where are we. Done:. How to use . fork. , and . join. to write a parallel algorithm. Why using divide-and-conquer with lots of small tasks is best. Dictionary ADT. : Arrays, Lists and . Trees. Kate Deibel. Summer 2012. June 27, 2012. CSE 332 Data Abstractions, Summer 2012. 1. Where We Are. Studying the absolutely essential ADTs of computer science and classic data structures for implementing them.