I turned Qwen3.8-27B Q2_64 + llama.cpp into a fully TypeSafe AI-compatible Jev-like system. OpenAI API still intact! World’s first Vision-enabled Jev-like model! <10 GB VRAM, 170 ms on an RTX 3090 and ~140 tok/s in chat. 76% vs. 88% Jev-1.13 Acc. on a diverse 22,000-request typed-decision benchmark
Yesterday I released Bonsai-Llama-Jev - a Jev-like Typed Decision inference system for locally operated, sovereign AI. In my new typed-decision-bench , covering more than 22,000 decisions across 175 use cases , it is currently the strongest open system I’ve tested, reaching around 76% Soft Accuracy…
Read the full story at r/LocalLLaMA ↗
Timeline · 1 report
- 2026-09-23 14:03 · r/LocalLLaMA
I turned Qwen3.8-27B Q2_64 + llama.cpp into a fully TypeSafe AI-compatible Jev-like system. OpenAI API still intact! World’s first Vision-enabled Jev-like model! <10 GB VRAM, 170 ms on an RTX 3090 and ~140 tok/s in chat. 76% vs. 88% Jev-1.13 Acc. on a diverse 22,000-request typed-decision benchmark
More stories
- Transformers now runs llama.cpp quants — Hugging Face Blog
- Success running Qwen 3.8 27B EXL3 on RTX 3060 + 5060 Ti — r/LocalLLM
- yandex/AliceAI-Foundation-80B-A3B-Base: Russian-developed competitor to Qwen 35B and DeepSeek V4 Flash — r/LocalLLaMA
- Is ChatGPT currently the best free AI for creating highly realistic images with simple, straightforward prompts? — r/OpenAI
- Andreessen Horowitz is launching an ‘academy’ with no homework and partnerships with Palantir, Google, and Meta — The Verge AI
- The bear can dance: Qwen 3.8 27B on one 3090 for 3 weeks — r/LocalLLaMA
- I trained a 360M-param Python model from scratch on two workstation GPUs and wrote up every step, including the bugs — r/learnmachinelearning
- Meta Shows AI Agents Aren’t Just For Businesses Anymore — Wall Street Journal Technology
Get the daily brief of stories like this at 6:30 every morning →