Temperature 0 Isn't Deterministic: Batch Invariance in LLM Inference
My snapshot test had one job: send a fixed prompt at temperature=0 , compare the answer to a saved string, fail on any diff. It passed on my laptop every time. In CI it failed on some runs and not others. Same prompt, same model, same seed, same everything. I spent an afternoon hunting a race condi…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-09-23 17:25 · DEV Community — Machine Learning
Temperature 0 Isn't Deterministic: Batch Invariance in LLM Inference