New paper proposes ensemble routing for cost-aware LLM inference
An arXiv paper introduces ensemble routing for prompt-adaptive, cost-aware LLM inference, aiming to balance response quality and inference cost across heterogeneous web-based service requests.
Read the full story at arXiv cs.AI ↗
Timeline · 1 report
- 2026-11-10 05:00 · arXiv cs.AI
Ensemble Routing for Prompt-Adaptive Cost-Aware LLM Inference