Test results, retests and practical guides for running Hermes Agent on local models. Every number comes from our own runs on the leaderboard.
Three questions cover most choices when you pick a local model for Hermes Agent: what works best overall, what is fastest while still reliable, and what fits a smaller GPU. Here are the measured answers from our 16 tested setups.
On 24 October 2026 we upgraded our test install from Hermes Agent v0.22.4 to v0.22.5 and reran every earlier llama.cpp setup, so the whole board runs on one version. Most setups moved by two points or less, which is within normal run-to-run noise.
LocalAgentBoard is a new, independent leaderboard that answers one practical question: which local model, quant and engine should you run Hermes Agent on? Every result comes from real agent runs on an NVIDIA DGX Spark, with the exact config published.
Is a Q4 quant good enough for agent work, or do you need Q8? We ran three models at several quants through the same Hermes Agent tasks on a DGX Spark. Q4 scored within run-to-run noise of Q8 every time, with less memory and more speed.
If you rerun the same local model on the same agent tasks, the score moves. On LocalAgentBoard it moves by about two points out of 30. So a 30/30 does not beat a 29/30, and the board picks its overall winner by speed instead.
It depends on your memory. On our DGX Spark tests, the Best overall pick is the fastest setup in the top group of scores, and there are separate picks for the fastest reliable setup and for machines with 24 GB or less. The current picks are always on the leaderboard homepage.
Each model, quant and engine combination runs the same 10 offline tasks inside the real Hermes Agent, 3 times each, on an NVIDIA DGX Spark. A run passes only if an automatic checker finds the correct result on disk.
Rerunning the same setup moves its score by up to about 2 of 30 because models sample their answers. Gaps of 2 points or less are within that noise.
Unofficial. Not affiliated with or endorsed by Nous Research. Hermes Agent is a trademark of its owner.