Main Contribution of this Research Work ID: 828594
Download Pdf The PPT/PDF document "HTC (High-Throughput Computing) for rela..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1 HTC (High-Throughput Computing) for rela
HTC (High-Throughput Computing) for relatively longrunning applications consisting of loosely-coupled tasks ¥HPC (High-Performance Computing) targets efÞcientlyprocessing tightly-coupled parallel tasks ¥DIC (Data-intensive Computing) mainly focuses oneffectively leveraging distributed storage systems andparallel processing frameworksMany-Task Computing (MTC) as a newcomputing paradigm [I. Raicu, I. Foster, and Y.Zhao, MTAGSÕ08]¥A very large number of tasks (millions or even billions)¥Relatively short per task execution times (sec to min)¥Data intensive tasks (i.e., tens of MB of I/O per second)¥A large variance of task execution times (i.e., rangingfrom hundreds of milliseconds to hours)¥Communication-intensive, however, not based on message Main Contribution of this Research Work¥MOHA (Many-task computing On HAdoop)framework which can effectively combine Many-TaskComputing technologies with the existing Big Dataplatform Hadoopdeveloped as one of Hadoop YARN applications¥transparently cohost into the ecosystemseamless integration of various techniques such as high-performance task dispatching, effective dynamic loadbalancing, data-intensive workload supportHadoop YARN Execution Model¥platform layer is responsible for resource management(Þrst-level scheduling)¥Resource Manager, Node Manager¥framework layer coordinates application execution(second-level scheduling)¥ApplicationMaster