AINewsnow

LoRA & DoRA: The Math, Memory, and Trade-offs

Every engineer who has ever tried to fine-tune a modern 27B or 70B parameter model knows the rude awakening of GPU memory arithmetic. You look at the raw model weights and think: “27 billion parameters stored in 16-bit brain floats is only 54 gigabytes. That easily fits on an 80 GB NVIDIA H100, rig…

Read the full story at DEV Community — Machine Learning ↗

Timeline · 1 report

  1. 2026-09-26 16:09 · DEV Community — Machine Learning
    LoRA & DoRA: The Math, Memory, and Trade-offs

More stories

  1. Palo Alto CEO says slowing down AI is ‘unrealistic’, extinction threat ‘extremely small’ — CNBC Technology
  2. The Ezra Klein Show: Jensen Huang Thinks A.I. Alarmism Has Gone Too Far — Hard Fork (NYT)
  3. How far behind Nvidia is Huawei? — Epoch AI
  4. Nvidia CEO Jensen Huang dismisses AI fears as 'distraction' — Semafor Technology
  5. Black Forest Labs Releases FLUX 3 Action: A 7B Open-Weights World Action Model That Tops RoboLab-120 — MarkTechPost
  6. R9V Update: Created and adopted KVA projections based on Deepseek V4.1 Flash + HySparse2/MiMo-V3 for Qwen3.8 Flash Next. This is a game changer for models that don't natively implement it. 1.45-1.85x speedup in prefill to 3k+ at a small deficit to perplexity. [2x R9700, 128GB DDR5] — r/LocalLLaMA
  7. Making the MiniMax H3 Video VAE 2x Faster — r/StableDiffusion
  8. Jensen Huang claims he doesn't know his own address, and so children forgetting how to basic math is ok — r/OpenAI

Get the daily brief of stories like this at 6:30 every morning →