Putting the question before the context took my local Qwen from 89% to 100% on a decision benchmark, and from ~400 ms to ~80 ms
Small, free finding. I use local Qwen models for typed decisions: a state plus a question with fixed allowed answers, and I read the probability of each answer from the logits of one forward pass instead of generating text. I used to build the prompt like you would for a human: situation first, the…
Read the full story at r/LocalLLaMA ↗