Deriving Muon From Mirror Descent: The Proximal Geometry Behind Orthogonalization
Muon’s orthogonalization has traditionally been understood through constrained optimization, such as Frank-Wolfe under spectral norm constraints. Yet Muon itself is an unconstrained optimizer. This post develops an alternative derivation from unconstrained Online Mirror Descent (OMD), showing how orthogonalization emerges naturally from a gradient-induced proximal geometry defined by the gradient’s Gram matrix. Under this geometry, the inverse metric cancels the gradient’s singular values, leaving only its orthogonal factor. This derivation further reveals that orthogonalization is a special case \(\alpha = \frac{1}{2}\) within a continuous family of spectral transformations.