Memory-Efficient Pre-training of a 1.11B LLM on a 6 GB Consumer GPU

 BAdam + CPU offload + BitNet ternary weights + tied embeddings + chunked cross-entropy. DOI: 10.5281/zenodo.22725431 Full write-up: https://huggingface.co/blog/RitishReal/1b-petrain-llm-in-4gb-vram

Read the full story at r/machinelearningnews ↗

Timeline · 1 report

2026-09-26 15:29 · r/machinelearningnews
Memory-Efficient Pre-training of a 1.11B LLM on a 6 GB Consumer GPU

More stories

Unsecured OpenAI agents posted 53 user images on the internet without the lab's knowledge — TechCrunch AI
OpenAI AI agent breached Australian government website, PM says — France 24 — Artificial Intelligence
OpenAI rogue agents targeted govt and varsity websites in US, Australia before Hugging Face hack: What we know — Mint AI
Qwen 3.8 Flash Next q4_k_m, 130k context, q8 cache on 16GB VRAM ann 64GB RAM, 15-20 t/s on 4080 — r/LocalLLaMA
Revealing the details of how OpenAI agents hacked Hugging Face — Hacker News Front Page
Cannot find MiniMaxH3VAEDecodeFast node — r/huggingface
viggle-turbo isn't just faster - for most prompts, it's just as good — r/StableDiffusion
I built a resumable HuggingFace/Civitai downloader that also sorts and manages your ComfyUI model folders — r/huggingface

Get the daily brief of stories like this at 6:30 every morning →