AINewsnow

Running 35B MoE models on a Tesla K80 (2014) at ~21 tok/s, in plain C

I've spent about a year on an inference engine that doesn't need modern hardware. It's called Pantheon (LLM engine) on top of Foundry (ML library), both in C with no outside libraries. The CUDA backend targets Kepler ( sm_37 ), so a Tesla K80 you can buy used for very little does real work. Measure…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-10-20 17:49 · r/LocalLLaMA
    Running 35B MoE models on a Tesla K80 (2014) at ~21 tok/s, in plain C

More stories

  1. SpaceX and Tesla boards are exploring potential merger — Financial Times AI
  2. Space X and Tesla exploring merger, CEO Elon Musk says — Axios AI+
  3. SpaceX quarterly revenue beats estimates as AI and Starlink grow — The Independent Tech
  4. Investors in Anticipation Over Musk’s Next Big SpaceX Deal — New York Times AI
  5. local LLM for cybersecurity/VAPT on 2× Tesla T4 GPUs (32GB VRAM total)? — r/AI_Agents
  6. Budget rig for local LLM inference + distillation: 2x RTX 3060 12GB vs 2x Tesla P100 16GB? — r/LocalLLM
  7. Practical use case for Tesla Roadster 2 — r/aivideo
  8. Yann Lecun: Tesla "Flying" Roadster Is A Marketing Stunt — r/singularity

Get the daily brief of stories like this at 6:30 every morning →