RTX 3090, RTX 5090, A100 and H100 on vLLM: the cheapest GPU flips with your answer time
I measured four GPUs serving the same 7B model on vLLM. Which one is cheapest per token has no single answer. It flips with how fast your answers must be. The setup Same model (Qwen2.5-7B-Instruct), same vLLM (0.27.1, default flags), same load on every card: prompts of about 512 tokens, 256-token a…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-11-07 19:50 · DEV Community — Machine Learning
RTX 3090, RTX 5090, A100 and H100 on vLLM: the cheapest GPU flips with your answer time
More stories
- Election backlash against data centers rattles AI stocks and expansion plans — TechCrunch AI
- Anthropic strikes $3.4bn deal for gas-fired generators to power AI data centres — Financial Times AI
- OpenAI to keep expanding AI data centers despite election results favoring restrictions — DigiTimes Asia
- Tencent AI chief warns AI will soon master biology, raising new safety risks — SCMP China Tech
- Hydra-PSIE expands 'Coeziune OM–IA' DOI registry to 39 published works — DEV Community — AI
- Developer's 48-Hour Field Notes Detail Free-Tier LLM Batch Job Failures — DEV Community — AI
- Apple's John Ternus reportedly pushing for faster AI progress beyond Siri upgrades — Techmeme
- US AI Safety Center Staff Now Report to White House, Raising Independence Concerns — AP News
Get the daily brief of stories like this at 6:30 every morning →