throughput doubled, latency up a little, errors flat, same traffic. what happened?
i said c. what am i missing self hosting is the only one of these that changes the serving path enough to double throughput, and slightly worse latency felt like what youd get on smaller hardware than the hosted vendor runs
Read the full story at r/MLQuestions ↗
Timeline · 1 report
- 2026-09-23 15:29 · r/MLQuestions
throughput doubled, latency up a little, errors flat, same traffic. what happened?
More stories
- GPT-6 Sol and Luna now available on AI Gateway — Vercel Blog
- Gemini 3.8 text-to-speech says hello — Google Gemini Blog
- Anthropic launches Claude Opus 5.5 with stricter safeguards for cybersecurity — The Verge AI
- Alibaba unveils new AI chip to challenge NVIDIA, plans Qwen models with up to 10 trillion parameters — Mint AI
- No Shirt, No Shoes, No Service: Amazon Blocks Meta’s Muse AI From Shopping — CNET AI
- Moonshot’s Kimi K3 lands on Amazon in key test for Chinese open-source AI revenue — South China Morning Post Tech
- How Benchling secured multi-tenant AI agents with Amazon Bedrock AgentCore — AWS Machine Learning Blog
- Meet the Data Agent in ChatGPT Work — OpenAI YouTube
Get the daily brief of stories like this at 6:30 every morning →