"Which models fit on my GPU?" : I made a tool that answers it for any Hugging Face model, including GGUF quants and RAM offload

I kept doing the same math every time a model came out, so I turned it into a tool. Pick your GPU (100 of them, from a PS5 to an H200), a context length and how many requests you serve at once. It tells you: - whether the model fits, in each format from BF16 down to GGUF Q3_K_M - which other models…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

1. 2026-10-15 18:43 · r/LocalLLM
   "Which models fit on my GPU?" : I made a tool that answers it for any Hugging Face model, including GGUF quants and RAM offload