Estimating RAM and VRAM for local LLMs: a size and quantization table
People ask me "will this model run on my box?" almost every week. Most of the time you don't need a benchmark to answer that — you need two numbers: how many parameters the model has, and how many bytes each parameter occupies at the precision you picked. Here's the back-of-the-envelope math I use,…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-11-17 10:03 · DEV Community — Machine Learning
Estimating RAM and VRAM for local LLMs: a size and quantization table