Muse benchmarks

Results as tablesRelease main-f03640150, as of 2026-11-13

A snapshot taken when this site was last built. The charts above read live data and may be newer. The same tables as markdown.

Throughput on Apple Silicon, tokens per second: every engine on the same model and machine, fastest first
Machine	Model	Setup	Engines (tok/s)
M5 Max 128 GB	Muse-Glimmer-30B	Single decode	Pie 32.6 · llama.cpp 29.4
M5 Max 128 GB	Muse-Glimmer-30B	Single prefill	Pie 903 · llama.cpp 541
M5 Max 128 GB	Muse-Glimmer-30B	Batched decode	Pie 274 · llama.cpp 120
M5 Max 128 GB	Muse-Glimmer-30B	Batched prefill	Pie 769 · llama.cpp 701
M5 Max 64 GB	Muse-Glimmer-30B	Single decode	Pie 33.2 · vLLM Metal 32.4 · llama.cpp 28
M5 Max 64 GB	Muse-Glimmer-30B	Single prefill	Pie 911 · vLLM Metal 759 · llama.cpp 465
M5 Max 64 GB	Muse-Glimmer-30B	Batched decode	Pie 249 · vLLM Metal 187 · llama.cpp 115
M5 Max 64 GB	Muse-Glimmer-30B	Batched prefill	vLLM Metal 811 · Pie 688 · llama.cpp 653
M4 Pro 48 GB	Muse-Glimmer-30B	Single decode	Pie 15.7 · vLLM Metal 15.6 · llama.cpp 14.5
M4 Pro 48 GB	Muse-Glimmer-30B	Single prefill	Pie 135 · vLLM Metal 124 · llama.cpp 120
M4 Pro 48 GB	Muse-Glimmer-30B	Batched decode	Pie 57 · vLLM Metal 51.9 · llama.cpp 46.6
M4 Pro 48 GB	Muse-Glimmer-30B	Batched prefill	Pie 131 · llama.cpp 128

Throughput on NVIDIA GPU, tokens per second: every engine on the same model and machine, fastest first
Machine	Model	Setup	Engines (tok/s)
B200 180 GB	Muse-Glimmer-30B	Single decode	Pie 102 · SGLang 95.8 · vLLM 87.9
B200 180 GB	Muse-Glimmer-30B	Single prefill	Pie 14,663 · SGLang 8,779 · vLLM 7,309
B200 180 GB	Muse-Glimmer-30B	Batched decode	Pie 12,979 · vLLM 12,602 · SGLang 7,343
B200 180 GB	Muse-Glimmer-30B	Batched prefill	SGLang 24,352 · vLLM 23,581 · Pie 22,238
RTX PRO 6000 96 GB	Muse-Glimmer-30B	Single decode	Pie 27.1 · SGLang 26 · vLLM 25.9
RTX PRO 6000 96 GB	Muse-Glimmer-30B	Single prefill	Pie 6,293 · SGLang 5,484 · vLLM 5,454
RTX PRO 6000 96 GB	Muse-Glimmer-30B	Batched decode	Pie 3,483 · vLLM 3,244 · SGLang 3,158
RTX PRO 6000 96 GB	Muse-Glimmer-30B	Batched prefill	Pie 7,499 · SGLang 6,966 · vLLM 6,958
H200 SXM 141 GB	Muse-Glimmer-30B	Single decode	Pie 73.1 · SGLang 65.8 · vLLM 63.6
H200 SXM 141 GB	Muse-Glimmer-30B	Single prefill	SGLang 11,588 · Pie 11,277 · vLLM 11,114
H200 SXM 141 GB	Muse-Glimmer-30B	Batched decode	SGLang 6,810 · vLLM 6,573 · Pie 6,356
H200 SXM 141 GB	Muse-Glimmer-30B	Batched prefill	SGLang 12,971 · Pie 12,922 · vLLM 12,324
H100 SXM 80 GB	Muse-Glimmer-30B	Single decode	Pie 51.6 · SGLang 49.4 · vLLM 48.5
H100 SXM 80 GB	Muse-Glimmer-30B	Single prefill	Pie 10,495 · SGLang 10,008 · vLLM 8,002
H100 SXM 80 GB	Muse-Glimmer-30B	Batched decode	SGLang 5,894 · vLLM 5,811 · Pie 5,611
H100 SXM 80 GB	Muse-Glimmer-30B	Batched prefill	SGLang 12,313 · Pie 11,990 · vLLM 11,863
A100 SXM 80 GB	Muse-Glimmer-30B	Single decode	Pie 31.1 · SGLang 28.7 · vLLM 28.7
A100 SXM 80 GB	Muse-Glimmer-30B	Single prefill	Pie 3,538 · SGLang 3,470 · vLLM 3,299
A100 SXM 80 GB	Muse-Glimmer-30B	Batched decode	Pie 2,490 · SGLang 2,354 · vLLM 2,345
A100 SXM 80 GB	Muse-Glimmer-30B	Batched prefill	SGLang 4,500 · vLLM 4,439 · Pie 4,287

Throughput on AMD GPU, tokens per second: every engine on the same model and machine, fastest first
Machine	Model	Setup	Engines (tok/s)
MI355X 288 GB	Muse-Glimmer-30B	Single decode	Pie 85 · vLLM 82.5 · SGLang 80.3
MI355X 288 GB	Muse-Glimmer-30B	Single prefill	vLLM 14,062 · Pie 13,478 · SGLang 12,736
MI355X 288 GB	Muse-Glimmer-30B	Batched decode	Pie 11,671 · SGLang 11,665 · vLLM 11,650
MI355X 288 GB	Muse-Glimmer-30B	Batched prefill	Pie 24,322 · SGLang 24,014 · vLLM 21,416
MI350X 288 GB	Muse-Glimmer-30B	Single decode	Pie 82.7 · vLLM 81 · SGLang 77.9
MI350X 288 GB	Muse-Glimmer-30B	Single prefill	vLLM 12,071 · Pie 11,603 · SGLang 11,060
MI300X 192 GB	Muse-Glimmer-30B	Single decode	Pie 63.9 · vLLM 61.1 · SGLang 51.8
MI300X 192 GB	Muse-Glimmer-30B	Single prefill	vLLM 5,907 · SGLang 5,871 · Pie 4,703
MI300X 192 GB	Muse-Glimmer-30B	Batched decode	Pie 3,853
MI300X 192 GB	Muse-Glimmer-30B	Batched prefill	vLLM 9,048 · Pie 9,023