How much does one LLM request cost on your own GPU? (51s of compute, $0.002 per request, self-hosted vs cloud API)
I priced a single LLM request. Not the monthly bill, one request: 60k input tokens, 50k of that cached, 2k output, 90% uptime. On the RTX 4070 Ti Super preset (500 W, 1000 prompt / 50 decode tok/s, $0.30/kWh): 51 seconds per request $0.0021 of electricity per request 1.54k requests per day $3.24 pe…
Read the full story at r/LocalLLM ↗
Timeline · 1 report
- 2026-10-29 09:37 · r/LocalLLM
How much does one LLM request cost on your own GPU? (51s of compute, $0.002 per request, self-hosted vs cloud API)