routing algorithms

Equifinality in Mixture of Experts: Routing Topology Does Not Determine Language Modeling Quality

This paper investigates whether routing topology actually determines language modeling quality in Mixture-of-Experts (MoE) architectures. The authors found that different routing variants, including a novel cosine-similarity based one, result in statistically equivalent asymptotic perplexity, suggesting that routing design has a smaller impact on final quality than previously thought.