Qwen3.6-35B-A3B with Hermes Agent

UD-Q6_K (GGUF) on llama.cpp commit a37cc46. Tested

Task completion

96.7% (29/30)

Tool-call success

75.7%

Decode tok/s

60.2

Prefill tok/s

2698.1

Server memory

28.4 GB

Avg turns

3.9

Other quants of this model

Quant	Score	Decode tok/s	Prefill tok/s	Server memory	Hermes
Q8_0 (GGUF)	30/30	58	2688.7	35.5 GB	v0.22.5
UD-Q6_K (GGUF) (this page)	29/30	60.2	2698.1	28.4 GB	v0.22.5
UD-Q4_K_M (GGUF)	29/30	67	3100.7	21.8 GB	v0.22.5

Same engine, same 10 tasks × 3 and the same context rules. Memory is the llama-server process at 64K context. Differences of a few points are within noise.

Setup

Model: Qwen3.6-35B-A3B
Family / maker: Qwen · Alibaba Qwen
Parameters: 35B (3B active)
Architecture: MoE
Weights: unsloth/Qwen3.6-35B-A3B-GGUF
Quant: UD-Q6_K (GGUF)
Engine: llama.cpp commit a37cc46
Hermes Agent: v0.22.5
Tested: 2026-10-24
Context: 192K for all 10 tasks
Memory tier: ≤128 GB (measured)

Per-task results

t01: 3/3 · full run 2026-10-24 (v0.22.4): 3/3 · day run 2026-10-22 (v0.22.4): 3/3
t02: 3/3 · full run 2026-10-24 (v0.22.4): 3/3 · day run 2026-10-22 (v0.22.4): 3/3
t03: 3/3 · full run 2026-10-24 (v0.22.4): 3/3 · day run 2026-10-22 (v0.22.4): 3/3
t04: 3/3 · full run 2026-10-24 (v0.22.4): 3/3 · day run 2026-10-22 (v0.22.4): 3/3
t05: 3/3 · full run 2026-10-24 (v0.22.4): 3/3 · day run 2026-10-22 (v0.22.4): 3/3
t06: 3/3 · full run 2026-10-24 (v0.22.4): 3/3 · day run 2026-10-22 (v0.22.4): 3/3
t07: 2/3 · full run 2026-10-24 (v0.22.4): 3/3 · day run 2026-10-22 (v0.22.4): 3/3
t08: 3/3 · full run 2026-10-24 (v0.22.4): 3/3 · day run 2026-10-22 (v0.22.4): 3/3
t09: 3/3 · full run 2026-10-24 (v0.22.4): 2/3 · day run 2026-10-22 (v0.22.4): 3/3
t10: 3/3 · full run 2026-10-24 (v0.22.4): 3/3 · day run 2026-10-22 (v0.22.4): 3/3

First number: full run (correctness + speed) 2026-10-24, Hermes Agent v0.22.5 (shown in the stats above).

Copy-paste config

1. Start the model server (llama.cpp)

Exact command used in our test. Download the GGUF from unsloth/Qwen3.6-35B-A3B-GGUF into ./models/.

llama-server --model ./models/Qwen3.6-35B-A3B-UD-Q6_K.gguf --alias qwen3-6-35b-a3b --host 127.0.0.1 --port 18080 --ctx-size 196608 --parallel 1 --n-gpu-layers 999 --flash-attn on --jinja

2. Point Hermes Agent at it

hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:18080/v1
hermes config set model.default qwen3-6-35b-a3b

Other Qwen setups

Qwen3.6-27B · Q6_K (GGUF) · llama.cpp
Qwen3.6-35B-A3B · Q8_0 (GGUF) · llama.cpp
Qwen3.6-35B-A3B · UD-Q4_K_M (GGUF) · llama.cpp
Qwen3-Coder-Next · FP8 · vLLM
Qwen3.5-122B-A10B · NVFP4 · vLLM
Qwen3.6-35B-A3B · NVFP4 (ModelOpt, safetensors) · vLLM