Batching product classification (17 categories) in one LLM request: how do you measure accuracy loss as batch size grows?
To save tokens and time, I want to pack as many items as possible into each request. Fewer requests means the instructions and category list are repeated less often, and the whole job finishes faster. The problem is that at some point, adding more items starts hurting accuracy (skipped items, wrong…
Read the full story at r/PromptEngineering ↗
Timeline · 1 report
- 2026-09-29 16:02 · r/PromptEngineering
Batching product classification (17 categories) in one LLM request: how do you measure accuracy loss as batch size grows?