Nemotron-Cascade-2-30B-A3B with Hermes Agent

Q8_0 (GGUF) on llama.cpp commit a37cc46. Tested

Task completion

93.3% (28/30)

Tool-call success

73%

Decode tok/s

57.6

Prefill tok/s

3115.4

Server memory

31.7 GB

Avg turns

5.1

Other quants of this model

Quant	Score	Decode tok/s	Prefill tok/s	Server memory	Hermes
Q4_K_M (GGUF)	29/30	71.5	3599	23.6 GB	v0.22.5
Q8_0 (GGUF) (this page)	28/30	57.6	3115.4	31.7 GB	v0.22.5

Same engine, same 10 tasks × 3 and the same context rules. Memory is the llama-server process at 64K context. Differences of a few points are within noise.

Setup

Model: Nemotron-Cascade-2-30B-A3B
Family / maker: Nemotron · NVIDIA
Parameters: 30B (3B active)
Architecture: MoE
Weights: bartowski/nvidia_Nemotron-Cascade-2-30B-A3B-GGUF
Quant: Q8_0 (GGUF)
Engine: llama.cpp commit a37cc46
Hermes Agent: v0.22.5
Tested: 2026-10-24
Context: 192K for all 10 tasks
Memory tier: ≤128 GB (measured)

Per-task results

t01: 3/3
t02: 3/3
t03: 3/3
t04: 3/3
t05: 2/3
t06: 3/3
t07: 3/3
t08: 3/3
t09: 2/3
t10: 3/3

Copy-paste config

1. Start the model server (llama.cpp)

Exact command used in our test. Download the GGUF from bartowski/nvidia_Nemotron-Cascade-2-30B-A3B-GGUF into ./models/.

llama-server --model ./models/nvidia_Nemotron-Cascade-2-30B-A3B-Q8_0.gguf --alias nemotron-cascade-2-30b-q8 --host 127.0.0.1 --port 18080 --ctx-size 196608 --parallel 1 --n-gpu-layers 999 --flash-attn on --jinja

2. Point Hermes Agent at it

hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:18080/v1
hermes config set model.default nemotron-cascade-2-30b-q8

Other Nemotron setups

Nemotron-3-Nano-30B-A3B · Q8_0 (GGUF) · llama.cpp
Nemotron-Cascade-2-30B-A3B · Q4_K_M (GGUF) · llama.cpp
Nemotron-3-Super-120B-A12B · NVFP4 · vLLM