AINewsnow

Run a 35B to 180B LLM Without a GPU: VIDRAFT POCKET Passes 3.15 Million Downloads

TL;DR VIDRAFT POCKET is a family of GGUF models that run large language models on hardware people already own: a PC with no GPU, a laptop, and (with the phone builds) an iPhone or Android device. It works with stock llama.cpp , Ollama, LM Studio and PocketPal. No fork, no CUDA, no cloud. As measure…

Read the full story at DEV Community — Machine Learning ↗

Timeline · 1 report

  1. 2026-11-01 02:12 · DEV Community — Machine Learning
    Run a 35B to 180B LLM Without a GPU: VIDRAFT POCKET Passes 3.15 Million Downloads

More stories

  1. How to Run LFM3-Omni-2B GGUF Locally (Only llama.cpp Makes It Talk Back) — DEV Community — Machine Learning
  2. A game scripting language that runs speech-to-text 5x faster than llama.cpp on CPU — Lobsters AI
  3. I wrote a Rust inference engine for Qwen models on a 12 GB GPU, decode is ~1.7x llama.cpp on my RTX3060 — r/LocalLLM
  4. A 2.6B model on an iPhone that always answers in valid JSON — DEV Community — Machine Learning
  5. NVIDIA RTX Spark: Grace CPU and Blackwell GPU on one package, up to 128GB shared memory — r/ArtificialInteligence
  6. Spark-X2.6-4B DSpark Is Here: More Than 2x Average BF16 Inference Speed — DEV Community — Machine Learning
  7. Which coding agent gets the most out of a local Qwen 3.8 27B? I lined up 10 and logged tokens, cache and sandbox blocks — r/AI_Agents
  8. I dpo post-trained llama-3.1-8b to be 100% more robust against jailbreaks (and measured what it costs) — r/huggingface

Get the daily brief of stories like this at 6:30 every morning →