Naiad: Incremental And Iterative Data-Parallel Computation

Channel:
Microsoft Research
Subscribers:
344,000
Published on ● Video Link: https://www.youtube.com/watch?v=lHBCq2AbWqc


Duration: 1:03:07
277 views
5

Naiad is a distributed system that supports efficient incremental and iterative data-parallel computation. It extends standard batch data-parallel processing models like MapReduce, Hadoop and Dryad/DryadLINQ to support efficient incremental updates in the manner of a stream processing system, while at the same time enabling arbitrarily nested fixed-point iteration. In this talk I describe the design and implementation of Naiad, and show that complex analyses like strongly connected components, requiring multiple nested loops, run on streams of input with subsecond update times.



Other Videos By Microsoft Research


2016-08-11ACID Objects And Modularity In The Cloud - Part 2 (B)
2016-08-11ACID Objects And Modularity In The Cloud - Part 2 (A)
2016-08-11Practical Consensus
2016-08-11Lattice-Based Cryptography
2016-08-11Reasoning with MAD Distributed Systems - Part 1 A
2016-08-11Reasoning with MAD Distributed Systems - Part 1 B
2016-08-11Concurrency and Consistency in Distributed Data Storage Systems - Part 1
2016-08-11Building Decentralized Systems Using DHTs : Part 3- Accountability
2016-08-11Transactional Memory - Semantics And Performance
2016-08-11Practical Byzantine Fault Tolerance
2016-08-11Naiad: Incremental And Iterative Data-Parallel Computation
2016-08-11Building Decentralized Systems Using DHTs - Part 2
2016-08-11Building decentralized systems using DHTs - Part 1
2016-08-11Transactional Memory - STM In The Small
2016-08-11Special Topic: AC ΓÇô Composable Asynchronous IO For Native Languages
2016-08-11Dynamically Checking Ownership Policies in Concurrent C/C++ Programs
2016-08-11Transactional Memory: Composability & Basic Algorithms
2016-08-11Elastic Replication for the Cloud
2016-08-11Barrelfish: A Study In Distributed Operating Systems On Multicore Architectures Part - 2
2016-08-11Barrelfish: A Study In Distributed Operating Systems On Multicore Architectures Part - 1
2016-08-11An Introduction To Distributed Computing


Tags:
microsoft research