We built our own inference engine so a 32B model could keep 262K context inside a 16 GB GPU budget. Here's an uncut run.
Hey r/LocalLLaMA , Blockway here, a small team from Hong Kong. After our v1 post, the most common question was basically "ok, but why does this model exist?" Totally fair. Here's our honest answer. The problem Long agent sessions eat VRAM. On a normal dense model every layer keeps a KV cache, and i…
Read the full story at r/LocalLLaMA ↗
Timeline · 1 report
- 2026-10-19 15:32 · r/LocalLLaMA
We built our own inference engine so a 32B model could keep 262K context inside a 16 GB GPU budget. Here's an uncut run.