GRIMOIRE: native C++/SYCL LLM inference for Intel Arc Pro B70 — Ornith 1.5 hits 175 tok/s single request, 464 tok/s at concurrency 8
https://preview.redd.it/azrcl8q94gth1.png?width=2172&format=png&auto=webp&s=20d4010874d090d1d35ee09a292e77696e4820b0 Hi everyone, I’ve been building GRIMOIRE , an experimental LLM inference engine for Intel Battlemage GPUs , with the Arc Pro B70 as its primary target. GRIMOIRE is written in C++ wit…
Read the full story at r/LocalLLM ↗