SIMPLE law predicts quadratic prefill cost
"prefill energy grows quadratically with the hidden dimension, so larger LLMs suffer most from long inputs;" [1] . That closed‑form observation collapses years of ad‑hoc profiling into a single analytic curve. It simultaneously predicts how much extra work each new token imposes on the KV‑cache and…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-11-15 06:00 · DEV Community — Machine Learning
SIMPLE law predicts quadratic prefill cost