I made 27b model run on gtx 1070 8gb at 96k context.
Did some kernel optimisations for sm61. Achieved measurable speedup. It runs at actually quite usable speeds.
Read the full story at r/LocalLLM ↗
Timeline · 1 report
- 2026-11-09 21:40 · r/LocalLLM
I made 27b model run on gtx 1070 8gb at 96k context.