AINewsnow

llmserver – one bash script for local LLMs with model security checks (llama.cpp / vLLM / Ollama)

I got tired of pulling models blind, so I built llmserver: a single bash script that runs local models on Linux. Each model gets its own OpenAI-compatible endpoint, plus a gateway, Open WebUI chat, hybrid RAG over your docs, and private SearXNG search. The part I spent the most time on is security:…

Read the full story at r/huggingface ↗

Timeline · 1 report

  1. 2026-10-27 14:43 · r/huggingface
    llmserver – one bash script for local LLMs with model security checks (llama.cpp / vLLM / Ollama)

More stories

  1. What's the best front-end for LLM assisted studying? — r/LocalLLM
  2. Qwen3.8-27B Q6 on one RTX 5090: 69 → 100 tok/s short, 35 → 64 tok/s at 120K context, just by leaving LM Studio for a native llama.cpp build (full benchmarks + recipe) — r/LocalLLM
  3. Introducing Zest: model server + agent + friendly GUI — r/LocalLLaMA
  4. I made Pumpkin, a graphical manager for llama.cpp for my girlfriend. It turned out well, so I put it on github — r/LocalLLaMA
  5. Merge-Based Vocabulary Extension Stalls at the Pre-Tokenizer: An English-Preserving Repair for Nepali, Tested in Continued Pretraining — arXiv cs.CL
  6. Nanbeige4.3-33B Q4 on a MacBook Pro M5 Max, mlx-lm vs llama.cpp vs Kapitsa, Prefill/Decode 512 to 32K range. — r/LocalLLaMA
  7. Thoughts on privacy issues with the likes of Meta Muse, Grok Bot, OpenAI Dots, etc.? — r/artificial
  8. Quad 3090 vs 256 M5 Ultra For Agentic Coding — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →