FARO optimizer proposal targets automated allocation of LLM training compute between optimizers
An arXiv paper introduces FARO, a function-space adaptive return-to-risk optimization method that allocates training work between optimizers, such as orthogonalized momentum for attention, via cross-block equal-risk-contribution budgeting.
Read the full story at arXiv cs.LG ↗
Timeline · 1 report
- 2026-11-02 05:00 · arXiv cs.LG
FARO: Function-Space Adaptive Return-to-Risk Optimization via Cross-Block Equal-Risk-Contribution Budget Allocation