AI Hippo

Search models, tools, and guides

· Article · TOOLCHAIN

Build a local coding model: stack, weights, VRAM bands

Stacks are cheap to change; model size locks hardware. Weights fitting VRAM is not the same as fitting your context. Check the API 100K+20K bill before buying a card.

Order: stack → model → hardware

Hardware is the expensive lock, so the page walks cheap-to-dear. Many people already have a Mac or a 24GB card — the wizard runs in reverse from memory. Start at the [local coding hub](/en/local-coding/).

Platform Stack
Desktop NVIDIA Ollama → llama.cpp → vLLM/SGLang (dedicated box)
Apple Silicon MLX first; llama.cpp / Ollama for GGUF portability
Laptop Never start with vLLM
IDE Point Continue / Aider / Cline at localhost

Do not start with vLLM on a laptop. Prefer MLX on Apple Silicon; use llama.cpp / Ollama when you need portable GGUF. Point the IDE at localhost and use the [Agent](/en/agent/) and [Toolchain](/en/toolchain/) boards for repos. Decision tree: [software stack](/en/local-coding/stack/).

Only downloadable coding weights that fit a consumer band

Do not name-heuristic Claude or GPT into “local coding.” The allowlist requires an open license, downloadable weights, and a Q4 footprint that matches a desk band. 480B / 284B MoE is an API or a rack, not a 24GB download. The on-site composite rewards long context and low list price — not SWE. Vendor SWE-Pro is not an independent harness.

Tables and same-job OpenRouter prices: [models](/en/local-coding/models/). Single-card case study: [Qwen3.8-27B review](/en/insights/qwen3-8-27b-review/).

VRAM bands: memory is the lock, not TFLOPS

Q4-class parameter ceiling by memory band Enlarge image
Bands are classes, not street quotes. Q4 ceilings are a conservative desk protocol, not architecture-exact GiB.
Band Typical class Max params (Q4 class) Workloads
16GB RTX 4060 / 5060 Ti 16GB class 8B Q4 class complete
24GB RTX 4090 24GB 32B Q4 class complete, agent
32GB RTX 5090 32GB class 40B Q4 class complete, agent, long
48GB 48GB workstation / dual-24 class 70B Q4 class complete, agent, long
80GB 80GB-class pro card 120B Q4 class complete, agent, long
128GB Mac Studio 128–192GB 180B Q4 class complete, agent, long
512GB Mac Studio 512GB 480B Q4 class complete, agent, long

Weights fitting the card ≠ target context fitting. KV cache grows with the window; 27B Q4 on 24GB is an 8K–32K coding-agent on-ramp, not native 262K. Apple unified memory prefers MLX; NVIDIA pairs with CUDA stacks. Mac Studio is expensive but operationally simple — not a free lunch. Protocol and do-not-buy list: [hardware](/en/local-coding/hardware/).

TCO: token stickers are zero; electricity and depreciation are not

Same-job API bill versus local “zero sticker” Enlarge image
100K in + 20K out. Local is electricity and depreciation, not a token invoice. API bars are OpenRouter primary listings, illustrative.

Illustrative break-even: 3-year capex plus ~40 hours/month at 250W and $0.15/kWh. You need N 100K+20K coding jobs per month to match a consumer card. Tab-complete often never breaks even — keep paying the API. Do not buy: 70B on ≤16GB; a 4090 expecting Coder 480B; H100 for personal SWE.

[choose-ai-infrastructure-guide](/en/insights/choose-ai-infrastructure-guide/) is API-host selection, not desk GPUs. Quantitative trade-offs live in the [local coding wizard](/en/local-coding/).

Sources

Evidence and actions

Evidence: Consumer GPUs and Mac Studio decisions are expensive; the site lacked an indexable local-coding decision chain.

  • Time window: registry v1 + OpenRouter snapshot
  • Observation count: 14
  • Source type: editorial_manual

Insights