AINewsnow

More stories

  1. Qwen 27B (IQ4_XS, Q8/Q4 KV) at full 262k context on a 16GB RX 9070 XT: 80+ t/s decode with Adaptive KV Streaming — r/LocalLLM
  2. How to Run NVIDIA Nemotron Dory 4B Locally (Stock llama.cpp Can't Load It Yet) — DEV Community — Machine Learning
  3. Qwen Ships Qwen3-VL 8B Locally, Seeing Images and Video in 5 GB — AlphaSignal
  4. Round 3: Tested 128K context on a 3090 with real code (Django) and a novel. Finding one thing is solid, counting is hopeless — r/LocalLLM
  5. Efforts to upstream adaptive MTP depth to llama.cpp abandoned — r/LocalLLM
  6. New quant types suggested: IQ2_K, IQ3_K, IQ4_K, IQ5_K and IQ6_K — r/LocalLLaMA
  7. Has anyone used EXO in x86 machines, or am I just better off using llama.cpp or dllama? — r/LocalLLM
  8. Developer Mert Cobanov Runs llama.cpp on a Jailbroken PS5 — AlphaSignal

Get the daily brief of stories like this at 6:30 every morning →