AINewsnow

Train Gemma 4 on Its Own Cut-Short Answers: Half the Tokens on an 8 GB RTX 3050

This story is from 2026-10-25. It is preserved in the archive; the latest stories are on the live feed.

This article provides a step by step guide to fine-tuning Gemma 4 E2B on an 8 GB NVIDIA RTX 3050 so that it thinks in 64 tokens, using answers the model wrote itself under llama.cpp's thinking cap. Every log, script and number is committed. https://github.com/xbill9/lazy-gemma On all 1,319 GSM8K te…

Read the full story at DEV Community — Machine Learning ↗

Timeline · 1 report

  1. 2026-10-25 15:12 · DEV Community — Machine Learning
    Train Gemma 4 on Its Own Cut-Short Answers: Half the Tokens on an 8 GB RTX 3050

More stories

  1. GitHub - vincalkr/DS41Engine: DeepSeek-V4.1-Flash 1017 on 1x-2x RX 7900 XTX (gfx1100, ROCm/HIP): max optimizations — r/LocalLLM
  2. High-Performance llama.cpp fork for Legacy GPUs: Kepler Support, CUDA Graphs & Windows NCCL! — r/LocalLLaMA
  3. NVIDIA's Dory 1.0 4B (looped reasoning model) on Apple Silicon with MLX: 101 tok/s at 1 loop, 69 at 2, on an M5 Max (4.2 GB 8-bit build) — r/LocalLLM
  4. Qwen 3.8 Flash Next (129 GB) is better than 2.6 bpw DeepSeek V4.1 Flash 1017 (187 GB + 203 GB Engram) — r/LocalLLaMA
  5. Qwen3.8-27B at 262k context on a 16 GB RX 9070 on Windows: ~80 t/s with achyudh's ROCm KV-streaming fork — r/LocalLLM
  6. Massive B70 spec-decoding improvement - Intel XMX matrix engines refactored by budiony · Pull Request #29864 · ggml-org/llama.cpp — r/LocalLLaMA
  7. Backup your quants! — r/LocalLLaMA
  8. I ran local AI inference (AMD GPU) for 30 days and measured every watt — here's the real cost vs cloud APIs — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →