AINewsnow

Running an LLM fully offline on Android with Kotlin coroutines

A 0.5B-parameter chat model, 4-bit quantized, is about 490 MB. That fits on any recent phone, and llama.cpp runs it on the CPU with no GPU, no server and no API key. The hard part is not the inference. It's making it feel like a normal Kotlin API inside a normal Android app. This post is about the…

Read the full story at DEV Community — AI ↗

Timeline · 1 report

  1. 2026-11-02 18:26 · DEV Community — AI
    Running an LLM fully offline on Android with Kotlin coroutines

More stories

  1. For those of you running both ComfyUI and local LLMs, how are you managing GPU resources? — r/StableDiffusion
  2. LFM3-1.7b seems impressive — r/LocalLLaMA
  3. 769 tok/s on a Single RTX 4090: Spark-X2.6-1.7B DSpark — DEV Community — Machine Learning
  4. Halving the KV Cache Doubles Capacity and Buys 1% of Speed — DEV Community — Machine Learning
  5. Are bespoke inference engines overhyped? My Strata experience on 2× R9700 — r/LocalLLaMA
  6. How we fit a 27B model on a 12 GB laptop GPU — DEV Community — Machine Learning
  7. Qwen3.6-35B-A3B on a laptop RTX 5050 8GB: ~1000 tok/s prefill and ~40-46 tok/s decode (config + steps) — r/LocalLLM
  8. Run a 35B to 180B LLM Without a GPU: VIDRAFT POCKET Passes 3.15 Million Downloads — DEV Community — Machine Learning

Get the daily brief of stories like this at 6:30 every morning →