A 4B model faster than a 395M encoder? 14 ms vs 31 ms (2.2x). Another 4B model: 150 ms (10x slower)
This is Mica v0.5, the next version after v0.1. It's a decision model, the same kind of thing as Jev, which has been getting a lot of attention lately. It's 4B (Qwen3.5-4B + LoRA) and it doesn't write text. You hand it a state, a question and a list of options, and it gives back a probability for e…
Read the full story at r/LocalLLaMA ↗
Timeline · 1 report
- 2026-10-23 17:35 · r/LocalLLaMA
A 4B model faster than a 395M encoder? 14 ms vs 31 ms (2.2x). Another 4B model: 150 ms (10x slower)