How are exactly are Query, Key and Value obtained during self-attention?
Andrew Ng says they are linear projections using pre-trained weight matrices. But how were those matrices trained exactly? What was the input, what was the target variable? I need to know EVERYTHING.
Read the full story at r/learnmachinelearning ↗
Timeline · 1 report
- 2026-10-25 17:06 · r/learnmachinelearning
How are exactly are Query, Key and Value obtained during self-attention?