POSPAN: Position-Constrained Span Masking for Language Model Pre-training
This story is from 2026-09-16. It is preserved in the archive; the latest stories are on the live feed.
arXiv:2609.16061v1 Announce Type: new Abstract: Span-level masked language modeling (MLM) has shown to be advantageous to pre-trained language models over the original single-token MLM, as entities/phrases and their dependencies are critical to language understanding. Previous works only consider s…
Read the full story at arXiv cs.LG ↗
Timeline · 1 report
- 2026-09-16 04:00 · arXiv cs.LG
POSPAN: Position-Constrained Span Masking for Language Model Pre-training