Do you remnber SETI@Home : HiveMind: Decentralized Branch-Train-Mix Sparse MoE Upcycling & Speculative Swarm Inference over Consumer GPUs via DiLoCo-QLoRA and BOINC - Starting on old RTX 2070 !!! Let's build together HiveMind@Home
TL;DR Project HiveMind is an open-source distributed training and inference framework (think SETI@home / Folding@home meets Petals and DiLoCo ) designed to aggregate idle consumer gaming GPUs (8 GB–12 GB VRAM) and local NVMe storage into a community-owned Sparse Mixture-of-Experts (MoE) cluster. In…
Read the full story at r/learnmachinelearning ↗