AINewsnow

Run LLMs on your own machine.

This story is from 2026-10-27. It is preserved in the archive; the latest stories are on the live feed.

✨Ollama Ollama with a GUI: discover, download and chat locally. ✨llama.cpp The engine almost everything else runs on. Plain C++, no dependencies. ✨vLLM The serving engine production runs on. PagedAttention and continuous batching, on your own GPU. ✨MLX Apple's array framework. On an M-series Mac it…

Read the full story at DEV Community — Machine Learning ↗

Timeline · 1 report

  1. 2026-10-27 08:18 · DEV Community — Machine Learning
    Run LLMs on your own machine.

More stories

  1. Nanbeige4.3-33B Q4 on a MacBook Pro M5 Max, mlx-lm vs llama.cpp vs Kapitsa, Prefill/Decode 512 to 32K range. — r/LocalLLaMA
  2. Update: MAFDET now runs on Apple Silicon. Qwen3-14B answers in 12 s instead of 33 on an M2 Pro — r/LocalLLM
  3. Introducing Zest: model server + agent + friendly GUI — r/LocalLLaMA
  4. I made Pumpkin, a graphical manager for llama.cpp for my girlfriend. It turned out well, so I put it on github — r/LocalLLaMA
  5. Qwen3.8-27B Q6 on one RTX 5090: 69 → 100 tok/s short, 35 → 64 tok/s at 120K context, just by leaving LM Studio for a native llama.cpp build (full benchmarks + recipe) — r/LocalLLM
  6. Merge-Based Vocabulary Extension Stalls at the Pre-Tokenizer: An English-Preserving Repair for Nepali, Tested in Continued Pretraining — arXiv cs.CL
  7. Quad 3090 vs 256 M5 Ultra For Agentic Coding — r/LocalLLM
  8. I created a benchmark to test local LLMs on your Snapdragon NPU (open source) — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →