Duration: 0:00

1,555 views

33

Kwangjun Ahn, Senior Researcher at Microsoft Research AI Frontiers, introduces Dion, a next-generation optimizer in the style of Muon that orthonormalizes only the top-r subspace via amortized power iteration. Dion retains Muon’s fast convergence while significantly reducing compute and communication, scaling efficiently with FSDP/TP for very large models.

Dion paper: https://arxiv.org/abs/2504.05295
Dion optimizer: https://github.com/microsoft/dion

This session aired on September 24, 2025, at Microsoft Research Forum, Season 2 Episode 1.

Register for the series to learn about future episodes: https://aka.ms/registerresearchforumYTs2e1
Continue watching this episode: https://aka.ms/researchforumYTs2e1
Explore all previous episodes: https://aka.ms/researchforumYTplaylist