I OOM'd on 100 Rows Because One File Was 590 MB
This story is from 2026-10-25. It is preserved in the archive; the latest stories are on the live feed.
My validator got SIGKILL'd on 100 rows. Not 100 million — a hundred. And I'd already done the responsible thing and capped how many files it reads. The cap bounded file count, not file size. One 590 MB parquet file defeated it, and 100 rows took down a 7 GB VM. This is from validating large agent-t…
Read the full story at DEV Community — AI ↗
Timeline · 1 report
- 2026-10-25 19:13 · DEV Community — AI
I OOM'd on 100 Rows Because One File Was 590 MB