A living catalog of local LLM setups, connected to their hardware, stack and results. Benchmarks tell you what fits. This tells you what works.
◉ Born from a r/LocalLLM thread · September 2026
This wiki is only as useful as the details people share. Take your time — every careful contribution makes the map clearer for everyone.
Capacity is the reported total for this setup. Multi-GPU and unified-memory layouts can behave differently.
Qwen3.8-27B (NVFP4 W4A16 + FP8 KV) served on 2x RTX 2080 Ti 22GB over NVLink via a vLLM TP2 fork. 196K context cap, MTP K=3, prefix caching on. Feeds a 1-orchestrator + 7-worker harness: ~1227 tok/s prefill and ~29 tok/s decode solo at 65K, ~192 tok/s aggregate at 8-way. Rejected AWQ/FP8 weights (72% EOS failure), ExLlamaV3 (3-6x slower), and PP=2 (infeasible on 2 GPUs).
Daily OCR of math- and table-heavy PDFs into LaTeX-correct Markdown and Word, using Qwen vision models on llama.cpp instead of a smaller model that misreads numbers.
At work, local models handle bounded tasks on sensitive data: document RAG and classifying material into schemas, while frontier models only help build the systems.
A four-DGX-Spark cluster under vLLM serves several concurrent agentic coding and research sessions, with unquantized weights and honest notes on concurrency limits.
A product team developed their agentic tooling against local inference on a Strix Halo and midrange GPUs, then retargeted the same harness to cloud inference once it worked.
A small research group built a deterministic local pipeline that chunks project data, finds it again with a multimodal RAG and matches it into each project's own report structure.
Aerospace work with a legal requirement to keep data in-house led to a custom Qwen document parser and RAG, after a stock model struggled with an 785-page federal regulation.
A site reliability engineer builds custom apps and a theming engine inside Dynatrace with local Qwen, using a per-repo manifest so the model only reads the code it actually needs.
Rather than chatting with a local model, this user runs local document Q&A, summarisation, translation and grammar correction on an M4 Pro, then built a desktop app to tie the pieces together.
A self-built server with 1TB of ECC RAM and a mix of used 3090s and P100s produced internal tools whose economic impact is reported at ten times the hardware cost.
A writer runs Qwen 3.8 27B locally on two RTX 5060 Ti cards with a model card fine-tuned only to edit books and detect plot holes, not to write.
A rule-forwarded inbox feeds a local workflow on an Nvidia Jetson that ingests invoices, receipts and cards, tracks spending and builds a contact and inventory database.
Choose the path that fits you. Both routes go through human review before anything is published.
