Frontier models write backtests that run and are wrong. We graded them on hidden data.
I've been building quant-engineering eval tasks for agents. The agent gets raw synthetic market data and a strategy spec, writes a backtest, and a deterministic grader re-runs that code on hidden data across 5 seeds. The data includes the usual real-world problems (ticker renames, splits, dividends…
Read the full story at r/learnmachinelearning ↗
Timeline · 1 report
- 2026-10-22 16:13 · r/learnmachinelearning
Frontier models write backtests that run and are wrong. We graded them on hidden data.