PPT-Investigation of Data Locality and Fairness in MapReduce

Zhenhua Guo Geoffrey Fox Mo Zhou Outline Introduction Data Locality and Fairness Experiments Conclusions MapReduce Execution Overview 3 Google File System Read input data Data locality. Optimization. Techniques. . Presented by . Preethi Rajaram. CSS 548 Introduction to Compilers . Professor Carol Zander. Fall 2012 . Why?. Processor Speed . -. increasing at a faster rate than the memory speed. A General Analytical Study. Yongkun. Li, . Patrick P. C. Lee. , John C. S. . Lui. , . Yinlong. Xu. The Chinese . University . of Hong . Kong. University of Science and Technology of China. . 1. SSD Storage. , Collective Communication, and Services. Oral Exam, . Bingjing. Zhang. Outline. MapReduce. MapReduce. Frameworks. Iterative . MapReduce. Frameworks. Frameworks Based on . MapReduce. and Alternatives. Zhenhua . Guo. , Geoffrey Fox, Mo Zhou. Outline. Introduction. Data Locality and Fairness. Experiments. Conclusions. MapReduce Execution Overview. 3. Google File System. Read input data. Data locality. Zhenhua . Guo. , Geoffrey Fox, Mo Zhou. Outline. Introduction. Analysis of Data Locality. Optimality of Data Locality. Experiments. Conclusions. MapReduce Execution Overview. 3. Google File System. Read input data. Simplified Data Processing on Large . Clusters. by Jeffrey Dean and Sanjay . Ghemawa. Presented by Jon Logan. Outline. Problem Statement / Motivation. An Example Program. MapReduce. . vs. Hadoop. GFS / HDFS. 1) Purlieus: Locality-aware Resource Allocation for . MapReduce. in a Cloud, SC 2011.. 2) Making Cloud Intermediate Data Fault-Tolerant, SOCC 2010. . Present by: Qiangju Xiao. Purlieus: Locality-aware Resource Allocation for. Zhenhua . Guo. PhD Thesis Proposal. Outline. Introduction and Motivation. Literature Survey. Research Issues and Our Approaches. Contributions. 2. Traditional HPC Architecture vs. the Architecture of Data Parallel Systems. Zhenhua . Guo. , Geoffrey Fox, Mo Zhou. Outline. Introduction. Analysis of Data Locality. Optimality of Data Locality. Experiments. Conclusions. MapReduce Execution Overview. 3. Google File System. Read input data. Yasin N. Silva and Jason Reed. Arizona State University. 1. This work is licensed under a Creative Commons Attribution-. NonCommercial. -. ShareAlike. 4.0 International License. See http://creativecommons.org/licenses/by-nc-sa/4.0/ for details.. Zhenhua . Guo. PhD Thesis Proposal. Outline. Introduction and Motivation. Literature Survey. Research Issues and Our Approaches. Contributions. 2. Traditional HPC Architecture vs. the Architecture of Data Parallel Systems. ”. Cathy O’Neil & Rachel . Schutt. , 2013. R & Hadoop. Compute squares. 2. R. # create a list of 10 integers. ints. <- 1:10. # equivalent to . ints. <- c(1,2,3,4,5,6,7,8,9,10). # compute the squares. Jimmy Lin. The iSchool. University of Maryland. Monday, March 30, 2009. This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States. See http://creativecommons.org/licenses/by-nc-sa/3.0/us/ for details. Source. MapReduce. : Simplified Data Processing in Large Clusters. . Jefferey. Dean and Sanjay . Ghemawat. . OSDI 2004. Example Scenario. 3. Genome data from roughly . one million users. 125 MB of data per user.