ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving

Researchers at Microsoft have developed ELDR, a new decode router for Mixture-of-Experts serving that improves performance by predicting expert activations and routing requests accordingly. ELDR reduces median TPOT by 5.9-13.9% compared to existing methods. This optimization could significantly impact the efficiency of large language model serving.

1sec.ai