Sure it takes half a day to a day for most requests and longer for the big ones
I average about 5-8tps for the first 50k or so context, then it dips to 3-5 for 50-100k before auto compacting. Here’s the thing though, I dont need it to go faster… I can prompt it, steer it as I see things, and do other things while it slowly cooks… I’m doing it on my gaming laptop with 8gb vram.…
Read the full story at r/LocalLLM ↗
Timeline · 1 report
- 2026-10-06 16:11 · r/LocalLLM
Sure it takes half a day to a day for most requests and longer for the big ones