AINewsnow

Running 35B MoE models on a Tesla K80 (2014) at ~21 tok/s, in plain C

I've spent about a year on an inference engine that doesn't need modern hardware. It's called Pantheon (LLM engine) on top of Foundry (ML library), both in C with no outside libraries. The CUDA backend targets Kepler ( sm_37 ), so a Tesla K80 you can buy used for very little does real work. Measure…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-21 02:00 · r/LocalLLM
    Running 35B MoE models on a Tesla K80 (2014) at ~21 tok/s, in plain C

More stories

  1. SpaceX merge with Tesla is on the Horizon — AI Supremacy
  2. Sure, let’s smoosh SpaceX and Tesla together, who cares — The Verge AI
  3. Tesla Optimus production starts, targets 20,000 robots weekly — r/singularity
  4. Calling all Nvidia Tesla P100 users! — r/LocalLLM
  5. Rockets, robots and risk: Musk’s $3.6tn gamble to merge Tesla and SpaceX — Financial Times AI
  6. ASML and other critical semiconductor equipment suppliers expedite shipments to Musk's Terafab — Tom's Hardware
  7. Tesla and SpaceX form board committees to explore $3.5T merger around AI — r/artificial
  8. SpaceX and Tesla boards explore potential merger — Financial Times AI

Get the daily brief of stories like this at 6:30 every morning →