How much vram do you leave available when starting a new chat?
I’ve been testing a lot of different models and settings but I’m struggling to consistently figure out how much extra vram I need to give myself. From my understanding, vram fills up as the chat progresses and the context is increasing. I currently have my context set to 44k for the model I’m testi…
Read the full story at r/LocalLLM ↗
Timeline · 1 report
- 2026-10-10 17:25 · r/LocalLLM
How much vram do you leave available when starting a new chat?