Exponential backoff with jitter: 64 of 100 LLM calls still failed
This story is from 2026-10-18. It is preserved in the archive; the latest stories are on the live feed.
Originally published at tensorust.com . TL;DR: I fired 100 LLM calls at once into a rate limiter that refills 10 requests a second, using the default retry policy of my own C++ library, llm-retry: exponential backoff, 3 tries, plus or minus 10% jitter. 64 of the 100 calls still failed , and the pro…
Read the full story at DEV Community — AI ↗
Timeline · 1 report
- 2026-10-18 23:23 · DEV Community — AI
Exponential backoff with jitter: 64 of 100 LLM calls still failed