Duration: 33:29

375 views

2

Lightweight containers technologies such as Docker are more and more adopted for deploying and managing micro-services architectures. Indeed, they hide a part of the complexity resulting from the language heterogeneity and the large choice of existing implementation frameworks that can be used to implement modern applications. Containers provide those benefits without introducing the general cost of the virtualization.

It exists lots of open-source configuration managers such as kubernetes, nomad, apache Mesos, or docker swarm to define and keep a standard modular applications in a consistent state and to dispatch its services on a cluster of remote nodes. Tools such as TaskCluster would complement the stack for integrating continuous integration and efficiently executing dependent tasks. However, there is still a lack of open-source solution for easily designing and distributing dataflow (i.e. a complex directed-acyclic-graph of tasks for processing data) on a distributed infrastructure to improve the performance of such applications.

In this talk, we will show how we can use Eclipse EMF and Sirius to build our own dataflow designer and integrate it with docker swarm. As a result, Eclipse becomes a platform that acts as an additional layer on top of docker swarm to design and execute complex distributed dataflow without specific Docker knowledge. We will demonstrate how we can use the resulting platform for cloud based video processing applications on top of a cluster of Raspberry Pis using Hypriot OS.

Olivier Barais, Benoit Combemale,Cedric Brun