Qwen3.6-35B-A3B with Hermes Agent
UD-Q4_K_M (GGUF) on llama.cpp commit a37cc46. Tested
Task completion
96.7% (29/30)
Tool-call success
77.4%
Decode tok/s
67
Prefill tok/s
3100.7
Server memory
21.8 GB
Avg turns
3.7
Other quants of this model
| Quant | Score | Decode tok/s | Prefill tok/s | Server memory | Hermes |
|---|---|---|---|---|---|
| Q8_0 (GGUF) | 30/30 | 58 | 2688.7 | 35.5 GB | v0.22.5 |
| UD-Q6_K (GGUF) | 29/30 | 60.2 | 2698.1 | 28.4 GB | v0.22.5 |
| UD-Q4_K_M (GGUF) (this page) | 29/30 | 67 | 3100.7 | 21.8 GB | v0.22.5 |
Same engine, same 10 tasks × 3 and the same context rules. Memory is the llama-server process at 64K context. Differences of a few points are within noise.
Setup
- Model
- Qwen3.6-35B-A3B
- Family / maker
- Qwen · Alibaba Qwen
- Parameters
- 35B (3B active)
- Architecture
- MoE
- Weights
- unsloth/Qwen3.6-35B-A3B-GGUF
- Quant
- UD-Q4_K_M (GGUF)
- Engine
- llama.cpp commit a37cc46
- Hermes Agent
- v0.22.5
- Tested
- 2026-10-24
- Context
- 192K for all 10 tasks
- Memory tier
- ≤24 GB (measured)
Per-task results
- t01: 3/3
- t02: 3/3
- t03: 3/3
- t04: 3/3
- t05: 3/3
- t06: 3/3
- t07: 2/3
- t08: 3/3
- t09: 3/3
- t10: 3/3
Copy-paste config
1. Start the model server (llama.cpp)
Exact command used in our test. Download the GGUF from unsloth/Qwen3.6-35B-A3B-GGUF into ./models/.
llama-server --model ./models/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf --alias qwen3-6-35b-a3b-q4 --host 127.0.0.1 --port 18080 --ctx-size 196608 --parallel 1 --n-gpu-layers 999 --flash-attn on --jinja2. Point Hermes Agent at it
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:18080/v1
hermes config set model.default qwen3-6-35b-a3b-q4Other Qwen setups
- Qwen3.6-27B · Q6_K (GGUF) · llama.cpp
- Qwen3.6-35B-A3B · UD-Q6_K (GGUF) · llama.cpp
- Qwen3.6-35B-A3B · Q8_0 (GGUF) · llama.cpp
- Qwen3-Coder-Next · FP8 · vLLM
- Qwen3.5-122B-A10B · NVFP4 · vLLM
- Qwen3.6-35B-A3B · NVFP4 (ModelOpt, safetensors) · vLLM