Zero Forgetting in LLMs? We Benchmarked Null-Space Projection on SmolLM and Hit the Plasticity Wall
This story is from 2026-10-29. It is preserved in the archive; the latest stories are on the live feed.
Why orthogonal projections eliminate catastrophic forgetting in autoregressive Transformers—and the mathematical reason behind the 62% downstream plasticity penalty. The Continual Learning Dilemma in LLMs Fine-tuning Large Language Models sequentially without retraining from scratch is notoriously…
Read the full story at DEV Community — AI ↗
Timeline · 1 report
- 2026-10-29 12:06 · DEV Community — AI
Zero Forgetting in LLMs? We Benchmarked Null-Space Projection on SmolLM and Hit the Plasticity Wall