Menu

← Leaderboard

Hermes-4-14B with Hermes Agent

Q4_K_M (GGUF) on llama.cpp commit a37cc46. Tested

Task completion
40% (12/30)
Tool-call success
56%
Decode tok/s
22.7
Prefill tok/s
1706.5
Server memory
18.4 GB
Avg turns
5.1

Setup

Model
Hermes-4-14B
Family / maker
Hermes · Nous Research
Parameters
14B
Architecture
dense
Weights
bartowski/NousResearch_Hermes-4-14B-GGUF
Quant
Q4_K_M (GGUF)
Engine
llama.cpp commit a37cc46
Hermes Agent
v0.22.4
Tested
2026-10-23
Context
64K (65,536 tokens); long-context task t09 at 128K or 192K
Memory tier
≤24 GB (measured)

Trained context is 40,960 tokens, below Hermes Agent's 64K minimum, so the server uses YaRN rope scaling (--rope-scaling yarn --rope-scale 2 --yarn-orig-ctx 32768, the method documented for its Qwen3-14B base) to serve 64K. Task t09 pastes a ~127K-token log, which does not fit this model's window, so Hermes rejects it and t09 counts as failed (0/3).

Per-task results

  • t01: 1/3
  • t02: 3/3
  • t03: 2/3
  • t04: 2/3
  • t05: 0/3
  • t06: 0/3
  • t07: 1/3
  • t08: 0/3
  • t09: 0/3 (does not fit: ~127K-token prompt vs 64K window)
  • t10: 3/3

Copy-paste config

1. Start the model server (llama.cpp)

Exact command used in our test. Download the GGUF from bartowski/NousResearch_Hermes-4-14B-GGUF into ./models/.

llama-server --model ./models/NousResearch_Hermes-4-14B-Q4_K_M.gguf --alias hermes-4-14b --host 127.0.0.1 --port 18080 --ctx-size 65536 --parallel 1 --n-gpu-layers 999 --flash-attn on --jinja --rope-scaling yarn --rope-scale 2 --yarn-orig-ctx 32768

2. Point Hermes Agent at it

hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:18080/v1
hermes config set model.default hermes-4-14b

Other Hermes setups