dOPSD: On-Policy Self-Distillation for Diffusion Language Models

Researchers propose dOPSD, a novel on-policy self-distillation method for diffusion language models. It improves mathematical reasoning and code generation by leveraging internal denoising trajectories. The approach outperforms supervised and on-policy baselines on Dream and LLaDA models.

1sec.ai