A Cool Generic Concurrency Primitive in Rust
717A few years ago, I built a concurrent hash map designed specifically to support highly concurrent reads: https://github.com/jonhoo/rust-evmap. It is part of what gives Noria, my PhD thesis work, such great read performance. The concurrency primitive that enables evmap to be so fast isn't really specific to maps though, and it's bothered me for a while that the particular datastructure I used it for in Noria was the only available way to use it.
In this stream, we explore an idea I came up with a few weeks ago to make a generic version of just the concurrency primitive that can then be re-used for other datastructures as well. You can see the outline of the plan here: https://github.com/jonhoo/rust-evmap/issues/45#issuecomment-723706112. This entailed slicing out just the concurrency stuff from evmap, and making it into its own reusable crate, and then reworking evmap to use that library. We got more or less all the way there, and you can see the final result in this PR: https://github.com/jonhoo/rust-evmap/pull/73.
0:00:00 Introduction
0:10:15 How evmap works
0:42:56 The change
0:53:58 Reorganizing the code
0:58:46 Stripping out the concurrency primitive
3:01:01 Biobreak
3:03:42 Make evmap use LeftRight
4:52:38 Fixing up evmap unit tests
5:15:15 Associated types vs. generic traits
5:18:36 Fixing up more evmap unit tests
5:21:47 Fixing up the full test suite
5:37:15 Making ReadHandle take &self
6:02:18 Outro
You can see the unsafe code guidelines comment I made about transmuting a type that contains T to ManuallyDrop T here: https://github.com/rust-lang/unsafe-code-guidelines/issues/35#issuecomment-731656729.
You can watch the live version with comments at https://www.youtube.com/watch?v=pZSDTiVEcxs
Other Videos By Jon Gjengset
| 2021-07-10 | Implementing Hazard Pointers in Rust (part 2) |
| 2021-06-26 | Implementing Hazard Pointers in Rust |
| 2021-06-13 | Lock-Free to Wait-Free Simulation in Rust (part 2) |
| 2021-05-22 | Lock-Free to Wait-Free Simulation in Rust |
| 2021-04-30 | Crust of Rust: Dispatch and Fat Pointers |
| 2021-04-02 | Crust of Rust: Atomics and Memory Ordering |
| 2021-03-13 | Crust of Rust: The Drop Check |
| 2021-02-20 | Crust of Rust: Subtyping and Variance |
| 2021-01-23 | Q&A January 2021 (now with cat) |
| 2020-12-12 | The Unsafe Chronicles: Exhibit A: Aliasing Boxes |
| 2020-11-21 | A Cool Generic Concurrency Primitive in Rust |
| 2020-11-14 | Crust of Rust: Sorting Algorithms |
| 2020-10-23 | Thesis: Partial State in Dataflow-Based Materialized Views |
| 2020-08-19 | Q&A August #2 2020 |
| 2020-08-09 | Q&A August 2020 |
| 2020-08-05 | Crust of Rust: Channels |
| 2020-07-25 | Thesis Talk: The Evaluation Chapter |
| 2020-06-17 | Crust of Rust: Smart Pointers and Interior Mutability |
| 2020-05-27 | Crust of Rust: Iterators |
| 2020-04-29 | Crust of Rust: Declarative Macros |
| 2020-04-01 | (Partially) fixing a bug in a Rust research database |