Nginx proxy_buffering Ate My LLM Stream: 0.4s First Token Became 6.1s
On my laptop the chat app typed like a person. Tokens dripped in 0.4 seconds after I hit enter. On the server, the same app sat there with a blinking cursor for six seconds, then dumped the whole answer on screen at once like someone pasting from Notepad. Same code. Same model. Same prompt. The onl…
Read the full story at DEV Community — AI ↗
Timeline · 1 report
- 2026-11-15 03:42 · DEV Community — AI
Nginx proxy_buffering Ate My LLM Stream: 0.4s First Token Became 6.1s