Evolution of Extract-Transform-Load (ETL) processes towards data product pipelines

Subscribers:
24,000
Published on ● Video Link: https://www.youtube.com/watch?v=fqsqPq5qdOE



Duration: 0:00
21 views
1


The rise of data as a first-class asset has led to creating infrastructures and tools designed to enhance organisations’ abilities to monetise them internally. One of the most powerful tools have been ETLs, which govern the internal data operations, assisting in companies in their quest to becoming data-driven. Lately, these horizons have expanded with the apparition of new ecosystems for data exchange, such as Data Spaces or initiatives like Gaia-X or SIMPL, allowing companies to monetise data externally, e.g. sharing or selling them. However, traditional ETLs fall short to serve this purpose. In this article we try to offer a technological comparison of how current ETL tools are prepared to address the new concept of data pipeline aimed at achieving a data product. Furthermore, this comparison is proposed within the framework of a project like DATAMITE, which allows it to be provided with real scenarios and use cases, in which its benefits and applicability can be accurately appreciated.




Other Videos By Eclipse Foundation


2024-12-11Theia IDE: The Why, the How and the Future
2024-12-10Open Source Facing European Regulation: What's the Plan? - OCX 2024
2024-12-10Eyes Wide Open, AIs Wide Open - Or How to Remain in Control in the Age of AI - OCX 2024
2024-12-10Enhancing Software Supply Chain Security: Approaches to Software Composition Analysis - OCX 2024
2024-12-10Zero Install Embedded CC++ Development Running GDB in the Browser with WebAssembly
2024-12-10TMLL Trace Server Machine Learning Library, Use AI for Trace Analysis
2024-12-10Listening, Learning, Leading: The Road to the Open Source AI Definition - OCX 2024
2024-12-09Christofer Dutz on Fostering Open Source in Industrial Automation
2024-12-08Francisco Carneiro on Why Joining the Eclipse Foundation and Open Source Benefits Businesses
2024-12-08What is next? Towards Dataspace Interoperability: Open standardisation and implementation
2024-12-05Evolution of Extract-Transform-Load (ETL) processes towards data product pipelines
2024-12-02Eclipse IDE November 2024 Community Call: Engage and Shape the Future of SWT
2024-12-02Industrial Toolchains in the Era of Generative AI - OCX 2024
2024-12-02Mastering Target Platform Migrations: Challenges, Resources, and Best Practices - OCX 2024
2024-12-02Eclipse Theia - News from the Next Gen Tools Platform - OCX 2024
2024-12-02Enhancing Custom IDEs with AI: Strategies, Use Cases, and Pitfalls - OCX 2024
2024-12-02Integrating AI with Domain-Specific Tools: Practical Insights and Techniques - OCX 2024
2024-12-02The State of the Eclipse Foundation - OCX 2024
2024-12-02Eclipse ThreadX: A Star is (Re)born - OCX 2024
2024-12-02Hello World+ projects to test and benchmark software composition analysis tools - OCX 2024
2024-12-01The future of cybersecurity, today: Free and open source tools for CRA compliance for SMEs - OCX 24