AINewsnow

torch.compile vs eager for training a 468M Llama-style model on four GPUs: step time, memory and compile cost

Does torch.compile make a small transformer train faster? I rented four GPUs on Runcrate today (2026-10-29), an RTX 5090, an L40S, an RTX A6000 and an H100 PCIe, trained the same 468M-parameter Llama-style model in eager mode and compiled, with PyTorch's SDPA attention and with the plain "eager" at…

Read the full story at DEV Community — Machine Learning ↗

Timeline · 1 report

  1. 2026-10-29 03:24 · DEV Community — Machine Learning
    torch.compile vs eager for training a 468M Llama-style model on four GPUs: step time, memory and compile cost

More stories

  1. I tested LFM2.5-24B-A2B vs Qwen3.6-35B-A3B on 75 fresh multi-step tool tasks. Qwen solved 91%, LFM2.5 49%, and LFM2.5 said "done" on most of its failures — r/LocalLLM
  2. What is the cheapest way to fine-tune Llama on an H100? — DEV Community — Machine Learning
  3. Nanbeige4.3-33B Q4 on a MacBook Pro M5 Max, mlx-lm vs llama.cpp vs Kapitsa, Prefill/Decode 512 to 32K range. — r/LocalLLaMA
  4. A 2013 Mac Pro runs a 35B model at 66 tok/s reading. The first benchmark I trusted was wrong by about 9x. — r/LocalLLM
  5. mtmd: enable video input support for LFM3 model family by tdakhran · Pull Request #31283 · ggml-org/llama.cpp — r/LocalLLaMA
  6. shura.cpp: a llama.cpp fork with a VRAM governor, so a 35B MoE can share one 16 GB card with the desktop you keep using — r/LocalLLM
  7. LiquidAI/LFM3-Omni-2B-GGUF — r/LocalLLaMA
  8. Qwen3.8-27B Q6 on one RTX 5090: 69 → 100 tok/s short, 35 → 64 tok/s at 120K context, just by leaving LM Studio for a native llama.cpp build (full benchmarks + recipe) — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →