Hadoop Architecture and its usage at Facebook

Channel:
Microsoft Research
Subscribers:
344,000
Published on ● Video Link: https://www.youtube.com/watch?v=tZW4GEDrIzs


Duration: 1:09:22
2,204 views
30

This talk introduces the origin of the Hadoop Project and a overview of Hadoop File System Architecture. Then it talks about the Hadoop environment at Facebook, the configuration of hardware and software in our Hadoop cluster, size and volume of datasets, characteristics of jobs and the processes we have built on top of Hadoop to keep the data pipeline alive and active.



Other Videos By Microsoft Research


2016-09-072009 eScience: Lessons from myExperiment: Research Objects for Data Intensive Research
2016-09-072009 eScience: Social and Semantic Computing to Support Citizen Science
2016-09-07Construction of a MODIS Scientific Data Reprojection & Reduction Pipeline in Windows Azure Platform
2016-09-072009 eScience: Tupelo: a Framework for E-Science Knowledge Spaces
2016-09-072009 eScience: Navigating the Complex Web of Chemistry Using ChemSpider
2016-09-07Nurtureshock: New Thinking About Children
2016-09-07Navigating across multi-scale biological systems using cognitive modeling and systems approaches
2016-09-072009 eScience: Low Power Amdahl Blades for Data Intensive Computing
2016-09-072009 eScience: Bayesian Network Modelling of the Factors Influencing the Development of Obesity
2016-09-072009 eScience: Using Multipartite Graphs for Recommendation and Discovery
2016-09-07Hadoop Architecture and its usage at Facebook
2016-09-072009 eScience: Performing Science in the Cloud: A Haplotype Phasing Case Study
2016-09-072009 eScience: Generalized eScience Collaboration through Sharepoint
2016-09-072009 eScience: Collaborative Data Analysis with Taverna Workflows
2016-09-072009 eScience: Understanding and Maturing the Data-Intensive Scalable Computing Storage Substrate
2016-09-072009 eScience: Integrating Streaming Data and Semantic Repositories
2016-09-072009 eScience: The Vedea Visualization Language
2016-09-07Growth and Phase Transitions in Isolated and Interacting Networks
2016-09-07Woodland Park brings the Zoo to you! - Giving Campaign Presentation
2016-09-072009 eScience: Provenir ontology: Towards a Framework for eScience Provenance Management
2016-09-072009 eScience: Facilitating Next Generation Data Intensive Science using Semantic Technologies


Tags:
microsoft research