News · Guide

Best local LLM for Hermes Agent: best overall, fastest and best for small machines

Published 25 October 2026

Three questions cover most choices when you pick a local model for Hermes Agent: what works best overall, what is fastest while still reliable, and what fits a smaller GPU. Here are the measured answers from our 16 tested setups.

All numbers below come from the leaderboard as of 25 October 2026: 10 Hermes Agent tasks × 3 tries, on an NVIDIA DGX Spark with llama.cpp. Memory is what the model server used with a 64K context. Speed is single-stream decode speed in tokens per second (tok/s). On your own GPU the speeds will differ, but the ranking between setups usually holds better than the absolute numbers. The cards on the homepage are recomputed whenever results change, so check them for the current picks.

Best overall: Nemotron-Cascade-2-30B-A3B Q4_K_M

Nemotron-Cascade-2-30B-A3B Q4_K_M: 29/30, 71.5 tok/s, 23.6 GB.

Nine setups score within two points of the top score, which is within run-to-run noise (see why ±2 points is a tie). Among those nine, Nemotron-Cascade-2 at Q4_K_M is the fastest, so it is our Best overall. It is a mixture-of-experts model with about 3B active parameters, which is why it is fast while also fitting in about 24 GB.

Close alternatives in the top group:

• Qwen3.6-35B-A3B UD-Q4_K_M: 29/30, 67.0 tok/s, 21.8 GB.
• Gemma 4 26B-A4B UD-Q4_K_M: 30/30, 49.8 tok/s, 17.7 GB.
• gpt-oss-120b MXFP4: 30/30, 53.4 tok/s, but 61.1 GB, so it needs a 64 GB+ or 128 GB machine.

Fastest reliable: gpt-oss-20b MXFP4

gpt-oss-20b MXFP4: 27/30, 79.4 tok/s, 12.5 GB.

“Reliable” here means a score of 90% or more. gpt-oss-20b is the fastest setup that clears that bar, and it is also the smallest. One caveat: its two Hermes Agent v0.22.5 reruns scored 30/30 and 28/30 against 27/30 on v0.22.4. That spread is a little more than our ±2 noise band, so the setup is marked “under review” and 27/30 stays the headline score while we check it.

Best for small machines: Gemma 4 26B-A4B UD-Q4_K_M

Gemma 4 26B-A4B UD-Q4_K_M: 30/30, 49.8 tok/s, 17.7 GB.

Among setups whose model server used 24 GB or less, Gemma 4 26B-A4B at Q4 has the best score. On a 24 GB card (RTX 3090 or 4090), it plus Nemotron-Cascade-2 Q4_K_M and Qwen3.6-35B-A3B UD-Q4_K_M are the three to try. For 16 GB, gpt-oss-20b at 12.5 GB is the measured option so far. The Best for 16/24/128 GB page groups every setup by memory.

What to avoid for agent work (for now)

Some well-known models do much worse at tool-using agent tasks on this test. Hermes-4-14B Q4_K_M scored 13/30, Hermes-4.3-36B Q6_K 22/30 at 7.1 tok/s, and Hermes-4-70B Q4_K_M 21/30 at 4.5 tok/s (under review after a 24/30 rerun). These are dense models and they are slow on the DGX Spark. Their scores reflect this task set and these settings, not general model quality.

How to use the picks

1. Pick the row that matches your memory.
2. Open the model page and copy the server and Hermes Agent config we used.
3. Check the per-task results on that page if one kind of task (for example long context, t09) matters most to you.

How every number is measured is on the methodology page.

Unofficial. Not affiliated with or endorsed by Nous Research. Hermes Agent is a trademark of its owner.