What a frozen encoder can and cannot learn: the same rule scored 0.42 as arithmetic and 0.94 written in words
Small lesson from a side project that I wish someone had told me before I spent an evening on it. I train a small head on top of a frozen ModernBERT-large encoder (the open Laya decision model) to answer typed questions: which label, yes or no, which score. Training only the head is cheap, and on t…
Read the full story at r/learnmachinelearning ↗
Timeline · 1 report
- 2026-09-24 17:21 · r/learnmachinelearning
What a frozen encoder can and cannot learn: the same rule scored 0.42 as arithmetic and 0.94 written in words