Gemma 4 31B with Hermes Agent
NVFP4 on vLLM. Not tested yet
Task completion
Not tested yet
Tool-call success
Not tested yet
Decode tok/s
Not tested yet
Prefill tok/s
Not tested yet
Server memory
Not tested yet
Avg turns
Not tested yet
Setup
- Model
- Gemma 4 31B
- Family / maker
- Gemma · Google
- Parameters
- 31B
- Architecture
- dense
- Weights
- Exact repo confirmed at test time
- Quant
- NVFP4
- Engine
- vLLM (version recorded at test time)
- Hermes Agent
- Version recorded at test time
- Tested
- Not yet
- Context
- 64K (planned)
- Memory tier
- ≤24 GB (estimate)
Per-task results
Results for each of the 10 benchmark tasks will be listed here once tested.
Copy-paste config
The exact vLLM launch command will be published once it has been verified in testing. We only publish configs we have run.
Point Hermes Agent at it
# Point Hermes Agent at a local OpenAI-compatible server
hermes config set model.provider custom
hermes config set model.base_url http://localhost:8000/v1
hermes config set model.default <served-model-id>
hermes -z "Reply exactly HERMES_OK"Generic Hermes Agent setup for any local OpenAI-compatible endpoint (from the Hermes Agent docs). Replace <served-model-id> with the id from curl -s http://localhost:8000/v1/models.
Other Gemma setups
- Gemma 4 26B-A4B · Q8_0 (GGUF) · llama.cpp