Distilling a decision API into a head on a frozen encoder with a calibrated fallback loop: numbers, the cache trick, and where it fails

Project write-up with numbers rather than claims. stuntd records typed decisions (choice / score / yes-no) an app asks an LLM for and trains, per decision site, the head Laya ships (type embedding, two transformer layers, a scorer over option markers) on a frozen 400M ModernBERT encoder. Temperatur…

Read the full story at r/deeplearning ↗

Timeline · 1 report

1. 2026-09-23 08:38 · r/deeplearning
   Distilling a decision API into a head on a frozen encoder with a calibrated fallback loop: numbers, the cache trick, and where it fails