Beyond IceBERT: Open-Data ModernBERT Encoders for Icelandic from 12M to 1B Parameters
arXiv:2610.36230v1 Announce Type: new Abstract: IceBERT, the reference encoder for Icelandic, found that a cleaned web crawl is enough to train a strong model and that starting from a multilingual model can help on some tasks. We revisit both findings with the ModernBERT architecture. We train Icel…
Read the full story at arXiv cs.CL ↗
Timeline · 1 report
- 2026-10-28 04:00 · arXiv cs.CL
Beyond IceBERT: Open-Data ModernBERT Encoders for Icelandic from 12M to 1B Parameters