PPT-DryadLINQ

Author : celsa-spraggs | Published Date : 2016-04-27

A System for GeneralPurpose Distributed DataParallel Computing Yuan Yu Michael Isard Dennis Fetterly Mihai Budiu Ú lfar Erlingsson Pradeep Kumar Gunda Jon Currey

Presentation Embed Code

Download Presentation

Download Presentation The PPT/PDF document "DryadLINQ" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

DryadLINQ: Transcript

A System for GeneralPurpose Distributed DataParallel Computing Yuan Yu Michael Isard Dennis Fetterly Mihai Budiu Ú lfar Erlingsson Pradeep Kumar Gunda Jon Currey Microsoft Research Silicon Valley. S. A. L. S. A. Team. salsaweb. /salsa. . Community Grids Laboratory, . Digital Science Center. Pervasive Technology Institute. Indiana University. Virtual Cluster provisioning via XCAT. Supports both . Mihai Budiu, MSR-SVC. LiveLabs. , March 2008. Goal. 2. The Dryad Project. http://research.microsoft.com/research/sv/dryad. Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks. Michael Isard, Mihai Budiu, Yuan Yu, Andrew Birrell, and Dennis Fetterly. S. A. L. S. A. Group. http://salsahpc.indiana.edu. Principal Investigator Geoffrey Fox. Project Lead Judy Qiu. Scott . Beason. , . Jaliya. . Ekanayake. , . Thilina. . Gunarathne. , . Jong. . Youl. Explore the applicability of Microsoft technologies to real world scientific domains with a focus on data intensive applications. Expect data deluge will demand . multicore. enabled data analysis/mining. Thilina. . Gunarathne. , . Tak. -Lon . Wu. Judy . Qiu. , Geoffrey Fox. School of Informatics, Pervasive . Technology Institute . Indiana University. Introduction. Forth Paradigm – Data intensive scientific discovery. Large-Scale Data-Processing . Applications with . Data Skew: . A . Case Study in . Product-Offer . Matching . Ekaterina . Gonina. UC Berkeley. Anitha. . Kannan. , John Shafer, . Mihai. . Budiu. Microsoft . Lenin Ravindranath. Pradeep Kumar . G. unda, Chandu Thekkath, Yuan Yu, Li Zhuang. Motivation. Resources are poorly managed in a data center. Computation. Storage. Redundant . computations. Wasting resources. S. A. L. S. A. Team. salsaweb. /salsa. . Community Grids Laboratory, . Digital Science Center. Pervasive Technology Institute. Indiana University. Virtual Cluster provisioning via XCAT. Supports both . MapReduce. on Multicore Clusters. UMBC, Maryland. Judy Qiu. http://salsahpc.indiana.edu . School . of Informatics and Computing. Pervasive Technology Institute. Indiana University. Important Trends. Oliver Williams, Mihai Budiu. Microsoft Research, Silicon Valley. With slides contributed by . Johnny . Lee, Jamie . Shotton. NASA Ames, February 14, 2011. Outline. Hardware overview. The body tracking pipeline. Need to get Processing closer to storage. Need to get tasks close to data. Rack locality: . Hadoop. Kill task and if a local slot is available: Quincy. Why?. Network is bad: gives horrible performance.

Download Document

Here is the link to download the presentation.
"DryadLINQ"The content belongs to its owner. You may download and print it for personal use, without modification, and keep all copyright notices. By downloading, you agree to these terms.