Best local models for Hermes Agent
Which open-weight models actually get real Hermes Agent tasks done on your own hardware? We run every setup through the same tests and publish the exact config that worked.
- Score = tasks completed out of 30 (10 real agent tasks × 3 tries).
- Speed = tokens per second while answering.
- Memory = what the model server used on the DGX Spark (128 GB).
12 of 18 setups tested so far on an NVIDIA DGX Spark with llama.cpp and Hermes Agent v0.22.4, 2026-10-22 to 2026-10-23. How we test →
Highest score of all 12 tested setups (ties broken by tool-call success, then speed). It also fits small machines: the model server used 24 GB or less.
Fastest answers (79.4 tok/s) among setups that score 90% or more.
Next-best score among setups whose model server used 24 GB or less; Best overall also fits.
Ranking
| Config | |||||
|---|---|---|---|---|---|
| 1 | Nemotron-Cascade-2-30B-A3B Q4_K_M (GGUF) · llama.cpp · 30B (3B active) DetailsTool-call success 81% · night run (correctness + speed) 2026-10-23 | 30/30Recommended | 70.4 | 23.6 GB | |
| 2 | gpt-oss-120b MXFP4 (official GGUF) · llama.cpp · 120B DetailsTool-call success 77.8% · night run (correctness + speed) 2026-10-23 | 30/30Recommended | 53.6 | 61.1 GB | |
| 3 | Qwen3.6-35B-A3B UD-Q6_K (GGUF) · llama.cpp · 35B (3B active) DetailsTool-call success 77.4% · day run (correctness) 2026-10-22 | 30/30Recommended | 60.2 | 28.4 GB | |
| 4 | Gemma 4 26B-A4B Q8_0 (GGUF) · llama.cpp · 26B (4B active) DetailsTool-call success 61.9% · day run (correctness) 2026-10-22 | 30/30Recommended | 46.5 | 26.9 GB | |
| 5 | Qwen3.6-27B Q6_K (GGUF) · llama.cpp · 27B DetailsTool-call success 77.9% · night run (correctness + speed) 2026-10-22 | 29/30Recommended | 9.5 | 24.5 GB | |
| 6 | GLM-4.7-Flash Q4_K_M (GGUF) · llama.cpp DetailsTool-call success 66.7% · night run (correctness + speed) 2026-10-23 | 28/30Recommended | 63.4 | 20.5 GB | |
| 7 | gpt-oss-20b MXFP4 (official GGUF) · llama.cpp · 20B DetailsTool-call success 77.5% · night run (correctness + speed) 2026-10-22 | 27/30Recommended | 79.4 | 12.5 GB | |
| 8 | Qwopus-GLM-18B Q4_K_M (GGUF, healed) · llama.cpp · 18B DetailsTool-call success 68.9% · night run (correctness + speed) 2026-10-23 | 27/30Recommended | 19.9 | 13.1 GB | |
| 9 | Nemotron-3-Nano-30B-A3B Q8_0 (GGUF) · llama.cpp · 30B (3B active) DetailsTool-call success 72.3% · night run (correctness + speed) 2026-10-23 · day run 2026-10-22: 26/30 | 25/30Usable | 56.9 | 31.7 GB | |
| 10 | Hermes-4-70B Q4_K_M (GGUF) · llama.cpp · 70B DetailsTool-call success 65.5% · night run (correctness + speed) 2026-10-23 | 21/30Usable | 4.5 | 59.5 GB | |
| 11 | Hermes-4.3-36B Q6_K (GGUF) · llama.cpp · 36B DetailsTool-call success 53.4% · day run (correctness) 2026-10-22 | 21/30Usable | 7.1 | 43.5 GB | |
| 12 | Hermes-4-14B Q4_K_M (GGUF) · llama.cpp · 14B DetailsTool-call success 56% · night run (correctness + speed) 2026-10-23 · long-context task t09: does not fit: ~127K-token prompt vs 64K window | 12/30Not reliable | 22.7 | 18.4 GB |
Copy config copies the exact llama-server command we ran plus the Hermes Agent settings. Verdicts: Recommended = 90% or more, Usable = 70% to under 90%, Not reliable = under 70%. Data updated 2026-10-23.
Not tested yet (6)
- Gemma 4 31B · NVFP4 · vLLM Not tested yet
- Mistral Small 4 119B · NVFP4 · vLLM Not tested yet
- Nemotron-3-Super-120B-A12B · NVFP4 · vLLM Not tested yet
- Qwen3-Coder-Next · FP8 · vLLM Not tested yet
- Qwen3.5-122B-A10B · NVFP4 · vLLM Not tested yet
- Qwen3.8 35B-A3B · NVFP4 · vLLM Not tested yet