AINewsnow

Our self-hosted inference cost per token only beat the API once the GPUs had night work.

Our API bill for two internal tools had crept up to around 9k a month, so in spring we bought a 4 card box, put a mid size Qwen on vLLM and moved everything over. The spreadsheet said our cost per token would be about a fifth of the API which is what got finance to sign off. Three months later I pu…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-07 21:13 · r/LocalLLM
    Our self-hosted inference cost per token only beat the API once the GPUs had night work.

More stories

  1. Qwen Flash Next on Single B200 or B300, any pointers ? — r/LocalLLM
  2. ~188k warm ~60–67 tok/s: Qwen3.8-Flash-Next NVFP4 with Strata on a single RTX PRO 4500 32GB + 64GB DDR5. — r/huggingface
  3. I mapped every major Qwen release from 2023 to 2026: 44 models, from Qwen-7B to the 2.4T open weights (with sources) — r/machinelearningnews
  4. Worth moving on from Qwen3.6 35B A3B UD on a gaming PC? — r/LocalLLM
  5. Qwen Image 2.1 Uncensored MCP — r/StableDiffusion
  6. A benchmark for LLMs playing Civilization V. GLM-5.3 is ahead of Opus-5.5, and Qwen-3.8-27B holds up surprisingly well. — r/LocalLLaMA
  7. VNCCS 3.2.0 Released with Qwen Image 2.1 and MiniMax H3 support! — r/StableDiffusion
  8. Reflection AI Is About to Release a US Open-Weight Model to Take On DeepSeek and Qwen — r/LocalLLaMA

Get the daily brief of stories like this at 6:30 every morning →