Benchmarking 6 Open LLMs on a 2-Machine RPC Cluster (Consumer GPUs, ~96GB VRAM)
This story is from 2026-10-22. It is preserved in the archive; the latest stories are on the live feed.
In the endless pursuit of "bigger, bigger, and even bigger," I finally reached this point: two machines directly connected via a local area network, using RPC scheduling to split and load the model layer by layer across both hosts. The reason is simple— I have a limited budget . Buying a profession…
Read the full story at DEV Community — AI ↗
Timeline · 1 report
- 2026-10-22 13:27 · DEV Community — AI
Benchmarking 6 Open LLMs on a 2-Machine RPC Cluster (Consumer GPUs, ~96GB VRAM)