Digit-logits-based classifier with llama.cpp
Although I do not completely understand what the big deal is with Jev, it reminded me of something I implemented in mt_llm (which is build around the great llama.cpp) a while back to use a relatively small LLM (e.g. 9B or even smaller) as a kind of classifier without having to rely on the grammar f…
Read the full story at r/LocalLLaMA ↗
Timeline · 1 report
- 2026-09-19 05:08 · r/LocalLLaMA
Digit-logits-based classifier with llama.cpp