gemma4: graph optimizations (massive gains on pp for E2B and E4B) by max-krasnyansky · Pull Request #31356 · ggml-org/llama.cpp
big prompt processing speedup for gemma E2B/E4B
Read the full story at r/LocalLLaMA ↗
More stories
- Anthropic releases FlashProve, an open-source prototype that cryptographically verifies an AI response came from a given model, not one that a router swapped in (Anthropic) — Techmeme
- NVIDIA Shrinks DeepSeek V4.1 Flash by 75 GiB With Almost No Quality Loss — AlphaSignal
- [llama.cpp fork]: QFN + [specific quant] + [ultra specific hardware config] = [incredible and irreproducible pp and tg] — r/LocalLLaMA
- Anthropic open-sources FlashProve, zkSNARK proofs for 400B Llama 4 Maverick inference (110 tok/s on 1 H100) — r/machinelearningnews
- SillyTavern LLM settings for an OpenAI-compatible API — DEV Community — AI
- Choosing the Right Base Model for Fine-Tuning LLMs: Strategy Before Execution — DEV Community — Machine Learning
- Running an LLM fully offline on Android with Kotlin coroutines — DEV Community — AI
- Qwen3.8-27B UD-Q4_K_XL@262k context(66 tok/s at that depth) on a single 3090: 138 tok/s sampled decode (157 greedy, 233 code, 382 parallel),KL divergence @160k 0.00472 (q8_0: 0.00467) — r/LocalLLM
Get the daily brief of stories like this at 6:30 every morning →