AINewsnow

A 2.6B model on an iPhone that always answers in valid JSON

Small language models are good at one thing that apps need all the time: read this text and fill these fields . A letter becomes sender, deadline and next steps; a voice note becomes a task list; a receipt becomes an expense entry. This post shows how we run that fully on an iPhone with llama.cpp a…

Read the full story at DEV Community — Machine Learning ↗

Timeline · 1 report

  1. 2026-10-31 13:26 · DEV Community — Machine Learning
    A 2.6B model on an iPhone that always answers in valid JSON

More stories

  1. Spark-X2.6-4B DSpark Is Here: More Than 2x Average BF16 Inference Speed — DEV Community — Machine Learning
  2. Which coding agent gets the most out of a local Qwen 3.8 27B? I lined up 10 and logged tokens, cache and sandbox blocks — r/AI_Agents
  3. I dpo post-trained llama-3.1-8b to be 100% more robust against jailbreaks (and measured what it costs) — r/huggingface
  4. Run TeleAI's Xing4.0-29B-A4B GGUF via ik_llama.cpp — Full native mHC/MLA runtime + HF converter — r/LocalLLM
  5. PSA: Upgrade Flash Next to DFlash2 if you haven't! +17% on average — r/LocalLLaMA
  6. Janas: a C inference engine that runs MoE models bigger than your RAM by streaming experts from NVMe, plus an almost complete OpenAI-compatible server, MCP tools, vision and speech (GPL-3.0) — r/LocalLLM
  7. I spent a year making old Tesla P100/V100 cards fast for local LLMs. PXA 4.0 is out today: up to 3x the decode and 8x the prompt speed of mainline llama.cpp on the same box, same files. Numbers, method and the losses inside. — r/LocalLLM
  8. Arc Pro B60 on WSL2 (Vulkan via Mesa Dozen) is 8 to 40 times slower than native Windows Vulkan, and Qwen3.8-27B segfaults inside Intel's LLVM. Anyone seen this? — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →