Running code LLMs locally guarantees 100% data privacy, zero cloud API costs, and air-gapped security. However, selecting the right model requires matching the model parameters and quantization format to your physical GPU VRAM. Loading a model that exceeds VRAM causes CPU memory offloading, dropping token generation speeds from 50+ tokens/sec to unworkable single digits.
| GPU Hardware | VRAM | Optimal Model | Quantization | Footprint | Context | 1-Click Launch Command |
|---|---|---|---|---|---|---|
| RTX 4060, RTX 3070, Apple M 16GB | 8 GB | Qwen 2.5 Coder 7B | Q4_K_M | 4.7 GB | 16K | ollama run qwen2.5-coder:7b |
| RTX 4070, RTX 3060 12GB | 12 GB | Qwen 2.5 Coder 14B | Q4_K_M | 9.0 GB | 32K | ollama run qwen2.5-coder:14b |
| RTX 4080, RTX 5080, 4060 Ti 16GB | 16 GB | Devstral Small 2 (24B) | Q4_K_M | 15.0 GB | 384K | ollama run devstral-small-2:24b |
| RTX 3090, RTX 4090, Apple M-Max 32GB+ | 24 GB | Qwen3-Coder 30B (MoE) Qwen 2.5 Coder 32B |
Q4_K_M | 19.0 GB 20.0 GB |
256K 32K |
ollama run qwen3-coder:30bollama run qwen2.5-coder:32b |
| Apple Silicon Ultra (64GB - 128GB Unified) | 64GB+ | DeepSeek-R1 / V3 (671B MoE) | Q4_K_M | ~40-70 GB | 64K | ollama run deepseek-r1:70b |
Excellent at single-function generation, unit test creation, and concise bug fixes. Lightweight footprint fits on virtually any modern developer laptop.
The undisputed sweet spot for mid-tier GPUs. Achieves near-frontier level code synthesis for daily development tasks and refactoring loops.
Engineered by Mistral with a massive 384K context window. Allows feeding entire repositories and complex multi-file test tracebacks simultaneously.
Matches commercial API models on SWE-bench and HumanEval benchmarks. Operates flawlessly with Aider in local architect mode with zero cloud telemetry.
# 1. Start model in Ollama
ollama run qwen2.5-coder:14b
# 2. Run Aider targeting local Ollama endpoint
aider --model ollama/qwen2.5-coder:14b