Study Links Attention and Representation Drift to Zero-Shot Forgetting in CLIP Fine-Tuning

A new paper introduces methods to diagnose zero-shot forgetting in fine-tuned CLIP models, finding attention drift characterizes adaptation while representation drift predicts transfer degradation without needing labeled test data.

Read the full story at arXiv cs.CV ↗

Timeline · 1 report

1. 2026-11-03 05:00 · arXiv cs.CV
   Diagnosing Zero-Shot Forgetting in CLIP Fine-Tuning: Attention Drift Characterizes Adaptation, Representation Drift Predicts Transfer Degradation