PPT-Etcetera! CMSC 491 Hadoop-Based Distributed Computing

Spring 2016 Adam Shook Agenda Advanced HDFS Features Apache Cassandra Cluster Planning Advanced HDFS Features Highly Available NameNode Highly Available NameNode feature eliminates SPOF. Uni processor computing can be called centralized computing brPage 3br mainframe computer workstation network host network link terminal centralized computing distributed computing A distributed system is a collection of independent computers interc System fault A characteristic of a software system that can lead to a system error For example failure to initialize a variable could lead to that variable having the wrong value when it is used Human error or mistake Human behavior that results in Functional Programming with OCaml. CMSC 330. 2. Review. Recursion is how all looping is done. OCaml can easily pass and return functions. CMSC 330. 3. The Call Stack in C/Java/etc.. void f(void) {. int x;. Origins & Applications. Christopher Smith. Xavier Stevens. John Carnahan. Original Map & Reduce. LISP. map f(x) [x. 0. , x. 1. , …, x. n. ]. yields: [f(x. 0. ), f(x. 1. ), …, f(x. n. )]. reduce f(x, y) [x. Janani C Krishnamani. CSC 8320. Fall . 2011. Outline. Introduction. Architecture models. System architectures. Communication Network architectures. Examples. Future Ideas. Introduction. What are distributed systems? . Ruoming. Jin. Welcome!. Instructor: . Ruoming. Jin. Office: 264 MCS Building. Email: . jin. AT cs.kent.edu. Office hour: Tuesdays and Thursdays (4:30PM to 5:30PM) or by appointment. TA: Lin Liu. Email: . May 23. nd. . 2012. Matt Mead, Cloudera. Hadoop Distributed File System (HDFS). Self-Healing, High Bandwidth Clustered Storage. MapReduce. Distributed Computing Framework. Apache Hadoop. is an open source platform for data storage and processing that is…. Set up a large number of machines all identically configured. Connect them to a high speed LAN. And to the Internet. Accept arbitrary jobs from remote users. Run each job on one or more nodes. Entire facility probably running mix of single machine and distributed jobs, simultaneously. Janani C Krishnamani. CSC 8320. Fall . 2011. Outline. Introduction. Architecture models. System architectures. Communication Network architectures. Examples. Future Ideas. Introduction. What are distributed systems? . Big data. Big Data . Explosion of information. Iot. Analytics. Not just SQL (Structured query language)but unstructured data. Transformation from a entity based data to transactional databases. Industry. Early Adopter: ASU - Intel Collaboration in Parallel and Distributed Computing Yinong Chen , Eric Kostelich , Yann -Hang Lee, Alex Mahalov , Gil Speyer, and Violet R. Syrotiuk 1 st NSF /TCPP Workshop on Parallel and Distributed Computing Education ( Pierre Riteau. Université de Rennes 1, IRISA. INRIA Rennes – Bretagne . Atlantique. Rennes, . France. Pierre.Riteau@irisa.fr. Introduction To. Sky. . Computing. IaaS. . clouds. On demand/elastic . Spring 2016. Adam Shook. What Is Hive?. Developed by Facebook and a top-level Apache project. A data warehousing infrastructure based on . Hadoop. Immediately makes data on a cluster available to non-Java programmers via SQL like queries. Dr. . Barsha. . Mitra. CSIS . Dept. , BITS Pilani, Hyderabad Campus. Introduction. Course ID: SS ZG526, Title: Distributed Computing. allows for flexibly sharing resources (e.g., files and multimedia documents) stored across network-wide computers.