Data Science with Rust - Arrow, DataFusion, and Ballista by Andy Grove

Channel:
United States Brooks Builds (brookzerker)
Subscribers:
7,960
Published on ● Video Link: https://www.youtube.com/watch?v=UnCaJFa13oo


Duration: 50:09
2,783 views
83

Andy Grove spoke at the Denver Rust meetup on 2020-10-20 about Data Science with Rust - Arrow, DataFusion, and Ballista.


https://www.meetup.com/Rust-Boulder-Denver/events/272996842/


Details

Data Science with Rust - Arrow, DataFusion, and Ballista

Andy will explain why Rust is ideally suited for building the next generation of distributed compute platforms that are necessary for modern data science and will give an update on the current status of the various related projects that he is involved in.

Apache Arrow (https://arrow.apache.org/) defines a language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware like CPUs and GPUs. The Arrow memory format also supports zero-copy reads for lightning-fast data access without serialization overhead.

DataFusion (https://docs.rs/datafusion/1.0.1/datafusion/), now part of the Arrow project, is an in-memory query engine implement in Rust that provides SQL and DataFrame APIs for querying CSV and Parquet files (as well as custom data sources).

Ballista (https://github.com/ballista-compute/ballista) is a distributed compute platform loosely modeled after Apache Spark and primarily implemented in Rust, that leverages Arrow and DataFusion.

Speaker: Andy Grove

Andy Grove is a PMC member of Apache Arrow, where he donated the initial Rust implementation as well as the DataFusion query engine and has more recently become a contributor to Apache Spark.



Other Videos By Brooks Builds (brookzerker)


2020-11-04Highlight: Adding iframes to the streamer so they can last longer
2020-11-03Highlight: Recording what we learned about web assembly into a wiki
2020-11-02Highlight: Learning web assembly with Rust (full stream archive)
2020-11-01Creating a Chat vs. Streamer game with Rust + GGEZ part 42
2020-10-31Creating a Chat vs. Streamer game with Rust + GGEZ part 41
2020-10-30Creating a Chat vs. Streamer game with Rust + GGEZ part 40
2020-10-29Creating a Chat vs. Streamer game with Rust + GGEZ part 39
2020-10-28Creating a Chat vs. Streamer game with Rust + GGEZ part 38
2020-10-27Creating a Chat vs. Streamer game with Rust + GGEZ part 37
2020-10-26Creating a Chat vs. Streamer game with Rust + GGEZ part 36
2020-10-25Data Science with Rust - Arrow, DataFusion, and Ballista by Andy Grove
2020-10-25Creating a Chat vs. Streamer game with Rust + GGEZ part 35
2020-10-24Creating a Chat vs. Streamer game in Rust and GGEZ part 34
2020-10-21Creating a Chat vs. Streamer game in Rust and GGEZ part 33
2020-10-20Creating a Chat vs. Streamer game in Rust + GGEZ part 32
2020-10-18Testing audio
2020-10-18Testing audio
2020-10-18Testing audio
2020-10-18Taking a look at a community members game of snake
2020-10-17Creating a Chat vs Streamer game in Rust + GGEZ part 30
2020-10-16Creating Collaborative Culture by Dan Levy