Unlocking Structured Pruning for Reasoning Language Models via Attention-Preserving MLP Trimming
arXiv:2610.19121v1 Announce Type: new Abstract: Structured pruning reduces LLM inference costs by removing model components to produce smaller dense models. However, existing methods can cause reasoning performance to collapse on long-form reasoning tasks such as AIME. We propose \textit{LayerTrim}…
Read the full story at arXiv cs.LG ↗
Timeline · 1 report
- 2026-10-15 04:00 · arXiv cs.LG
Unlocking Structured Pruning for Reasoning Language Models via Attention-Preserving MLP Trimming