OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers

The OmniOpt project presents a unified framework for optimizer selection in large-scale model training, combining meta-pipeline transformations, norm-constrained linear minimization oracles, and a cross-domain benchmark. This provides the research community with an operational coordinate system for selecting optimizers under explicit mechanism and objective assumptions.

1sec.ai