AINewsnow

I built a GPT-2 KV cache from scratch: the tests that passed with a bug, and the memory numbers I got wrong

I started a blog to learn the maths behind AI/ML systems through implementations and experiments. This is my first post: GPT-2 small and a KV cache in PyTorch, measured on an NVIDIA A40 (fp32, TF32 off, batch size 1). I loaded pretrained GPT-2 weights, checked logits against Hugging Face, compared…

Read the full story at r/learnmachinelearning ↗

Timeline · 2 reports

  1. 2026-10-18 08:23 · r/MachineLearning
    GPT-2 KV cache from scratch: correctness, latency, and memory measurements [P]
  2. 2026-10-18 08:18 · r/learnmachinelearning
    I built a GPT-2 KV cache from scratch: the tests that passed with a bug, and the memory numbers I got wrong

More stories

  1. GPT-2 KV caching experiment: 1,000x fewer estimated FLOPs, about 6x lower per-token latency — r/deeplearning
  2. MossImage quants & SFT fix — r/StableDiffusion
  3. An Nvidia insider-turned-VC on why robotics hasn’t hit its ChatGPT moment, yet — TechCrunch AI
  4. Repeat with me: guardrails protect what my agent says. What my agent does needs a check outside the model. — r/AI_Agents
  5. NVIDIA RTX Spark Laptops Launch Globally With Local AI Agent Support — NVIDIA Blog
  6. RTX Spark Ignites Global Celebration as New Windows PCs Arrive — NVIDIA Blog
  7. Gemini 4 Argon rollout draws criticism over restrictive usage limits — r/singularity
  8. OpenAI introduces shared team task features for ChatGPT — OpenAI YouTube

Get the daily brief of stories like this at 6:30 every morning →