AMD GPU benchmarks

Results as tablesRelease main-f03640150, as of 2026-11-13

A snapshot taken when this site was last built. The charts above read live data and may be newer. The same tables as markdown.

Throughput on AMD GPU, tokens per second: every engine on the same model and machine, fastest first
Machine	Model	Setup	Engines (tok/s)
MI355X 288 GB	Qwen3.5-0.8B	Single decode	Pie 751 · SGLang 718 · vLLM 546
MI355X 288 GB	Qwen3.5-0.8B	Single prefill	Pie 80,944 · vLLM 24,473 · SGLang 20,517
MI355X 288 GB	Qwen3.5-0.8B	Batched decode	SGLang 58,959 · Pie 41,995 · vLLM 32,585
MI355X 288 GB	Qwen3.5-0.8B	Batched prefill	Pie 323,319 · SGLang 308,832 · vLLM 88,270
MI355X 288 GB	Qwen3-0.6B	Single decode	SGLang 665 · vLLM 631 · Pie 558
MI355X 288 GB	Qwen3-0.6B	Single prefill	Pie 91,340 · vLLM 35,877 · SGLang 34,159
MI355X 288 GB	Qwen3-0.6B	Batched decode	SGLang 64,176 · Pie 59,280 · vLLM 50,204
MI355X 288 GB	Qwen3-0.6B	Batched prefill	SGLang 495,664 · Pie 450,305 · vLLM 146,985
MI355X 288 GB	Gemma-4-E4B	Single decode	Pie 235 · vLLM 210 · SGLang 176
MI355X 288 GB	Gemma-4-E4B	Single prefill	Pie 36,428 · vLLM 21,190 · SGLang 15,262
MI355X 288 GB	Gemma-4-E4B	Batched decode	Pie 33,242 · vLLM 27,777 · SGLang 27,269
MI355X 288 GB	Gemma-4-E4B	Batched prefill	Pie 106,223 · SGLang 101,060 · vLLM 81,764
MI355X 288 GB	Qwen3.6-27B	Single decode	Pie 93 · vLLM 84.3 · SGLang 82.7
MI355X 288 GB	Qwen3.6-27B	Single prefill	Pie 11,244 · vLLM 10,554 · SGLang 9,606
MI355X 288 GB	Qwen3.6-27B	Batched decode	vLLM 6,267 · SGLang 6,204 · Pie 5,706
MI355X 288 GB	Qwen3.6-27B	Batched prefill	SGLang 23,034 · Pie 20,333 · vLLM 20,078
MI355X 288 GB	GPT-OSS-20B	Single decode	SGLang 444 · vLLM 366 · Pie 358
MI355X 288 GB	GPT-OSS-20B	Single prefill	vLLM 51,812 · Pie 30,269 · SGLang 18,973
MI355X 288 GB	GPT-OSS-20B	Batched decode	SGLang 43,649 · vLLM 30,460 · Pie 22,617
MI355X 288 GB	GPT-OSS-20B	Batched prefill	SGLang 133,334 · vLLM 80,988 · Pie 68,276
MI355X 288 GB	GPT-OSS-120B	Single decode	SGLang 321 · vLLM 247 · Pie 245
MI355X 288 GB	GPT-OSS-120B	Single prefill	vLLM 30,518 · Pie 17,185 · SGLang 11,863
MI355X 288 GB	GPT-OSS-120B	Batched decode	SGLang 30,132 · vLLM 21,092 · Pie 13,619
MI355X 288 GB	GPT-OSS-120B	Batched prefill	SGLang 116,011 · vLLM 53,707 · Pie 42,272
MI355X 288 GB	Qwen3.6-35B	Single decode	SGLang 276 · Pie 273 · vLLM 169
MI355X 288 GB	Qwen3.6-35B	Single prefill	Pie 22,703 · SGLang 9,868 · vLLM 8,539
MI355X 288 GB	Qwen3.6-35B	Batched decode	SGLang 17,564 · vLLM 13,895 · Pie 11,278
MI355X 288 GB	Qwen3.6-35B	Batched prefill	SGLang 90,253 · Pie 72,563 · vLLM 33,737
MI355X 288 GB	Muse-Glimmer-30B	Single decode	Pie 85 · vLLM 82.5 · SGLang 80.3
MI355X 288 GB	Muse-Glimmer-30B	Single prefill	vLLM 14,062 · Pie 13,478 · SGLang 12,736
MI355X 288 GB	Muse-Glimmer-30B	Batched decode	Pie 11,671 · SGLang 11,665 · vLLM 11,650
MI355X 288 GB	Muse-Glimmer-30B	Batched prefill	Pie 24,322 · SGLang 24,014 · vLLM 21,416
MI355X 288 GB	Gemma-4-26B	Single decode	Pie 216 · vLLM 186 · SGLang 186
MI355X 288 GB	Gemma-4-26B	Single prefill	Pie 24,712 · SGLang 15,823 · vLLM 15,106
MI355X 288 GB	Gemma-4-26B	Batched decode	vLLM 22,488 · Pie 20,411 · SGLang 15,253
MI355X 288 GB	Gemma-4-26B	Batched prefill	Pie 71,822 · SGLang 67,646 · vLLM 62,748
MI355X 288 GB	Gemma-4-31B	Single decode	Pie 80.4 · vLLM 73.9 · SGLang 72.2
MI355X 288 GB	Gemma-4-31B	Single prefill	Pie 12,234 · vLLM 12,037 · SGLang 11,886
MI355X 288 GB	Gemma-4-31B	Batched decode	vLLM 5,965 · Pie 4,842 · SGLang 3,685
MI355X 288 GB	Gemma-4-31B	Batched prefill	SGLang 18,591 · Pie 18,210 · vLLM 17,599
MI355X 288 GB	Qwen3.8-27B	Single decode	Pie 93.1 · vLLM 84.2 · SGLang 82.7
MI355X 288 GB	Qwen3.8-27B	Single prefill	Pie 11,326 · vLLM 10,430 · SGLang 9,564
MI355X 288 GB	Qwen3.8-27B	Batched decode	vLLM 6,275 · SGLang 6,170 · Pie 5,727
MI355X 288 GB	Qwen3.8-27B	Batched prefill	SGLang 20,599 · vLLM 19,805 · Pie 19,559
MI355X 288 GB	Qwen3.5-9B	Single decode	Pie 252 · SGLang 233 · vLLM 217
MI355X 288 GB	Qwen3.5-9B	Single prefill	Pie 30,068 · vLLM 15,707 · SGLang 15,151
MI355X 288 GB	Qwen3.5-9B	Batched decode	SGLang 19,432 · vLLM 18,307 · Pie 16,037
MI355X 288 GB	Qwen3.5-9B	Batched prefill	SGLang 73,646 · Pie 70,165 · vLLM 62,747
MI350X 288 GB	Qwen3.5-0.8B	Single decode	Pie 714 · SGLang 649 · vLLM 520
MI350X 288 GB	Qwen3.5-0.8B	Single prefill	Pie 67,237 · vLLM 23,457 · SGLang 19,527
MI350X 288 GB	Qwen3-0.6B	Single decode	SGLang 617 · vLLM 602 · Pie 539
MI350X 288 GB	Qwen3-0.6B	Single prefill	Pie 83,807 · vLLM 33,787 · SGLang 32,997
MI350X 288 GB	Gemma-4-E4B	Single decode	Pie 228 · vLLM 204 · SGLang 169
MI350X 288 GB	Gemma-4-E4B	Single prefill	Pie 32,734 · vLLM 18,815 · SGLang 14,895
MI350X 288 GB	Qwen3.6-27B	Single decode	Pie 90.9 · vLLM 82.3 · SGLang 80.2
MI350X 288 GB	Qwen3.6-27B	Single prefill	vLLM 10,066 · Pie 9,753 · SGLang 9,221
MI350X 288 GB	GPT-OSS-20B	Single decode	SGLang 422 · vLLM 350 · Pie 341
MI350X 288 GB	GPT-OSS-20B	Single prefill	vLLM 46,013 · Pie 27,206 · SGLang 17,014
MI350X 288 GB	GPT-OSS-120B	Single decode	SGLang 306 · vLLM 236 · Pie 233
MI350X 288 GB	GPT-OSS-120B	Single prefill	vLLM 25,435 · Pie 15,228 · SGLang 11,543
MI350X 288 GB	Qwen3.6-35B	Single decode	SGLang 265 · Pie 264 · vLLM 162
MI350X 288 GB	Qwen3.6-35B	Single prefill	Pie 20,896 · SGLang 9,465 · vLLM 8,496
MI350X 288 GB	Muse-Glimmer-30B	Single decode	Pie 82.7 · vLLM 81 · SGLang 77.9
MI350X 288 GB	Muse-Glimmer-30B	Single prefill	vLLM 12,071 · Pie 11,603 · SGLang 11,060
MI350X 288 GB	Gemma-4-26B	Single decode	Pie 206 · vLLM 181 · SGLang 179
MI350X 288 GB	Gemma-4-26B	Single prefill	Pie 22,879 · vLLM 17,483 · SGLang 16,370
MI350X 288 GB	Gemma-4-31B	Single decode	Pie 78.8 · vLLM 72.3 · SGLang 70.3
MI350X 288 GB	Gemma-4-31B	Single prefill	vLLM 10,181 · Pie 10,109 · SGLang 9,937
MI350X 288 GB	Qwen3.8-27B	Single decode	Pie 91 · vLLM 82.4 · SGLang 80.4
MI350X 288 GB	Qwen3.8-27B	Single prefill	vLLM 10,164 · Pie 9,755 · SGLang 9,635
MI350X 288 GB	Qwen3.5-9B	Single decode	Pie 246 · vLLM 211 · SGLang 190
MI350X 288 GB	Qwen3.5-9B	Single prefill	Pie 27,697 · vLLM 14,814 · SGLang 14,085
MI300X 192 GB	Qwen3.5-0.8B	Single decode	Pie 673 · SGLang 572 · vLLM 507
MI300X 192 GB	Qwen3.5-0.8B	Single prefill	Pie 42,602 · SGLang 14,059 · vLLM 12,914
MI300X 192 GB	Qwen3.5-0.8B	Batched decode	Pie 30,823 · SGLang 24,012 · vLLM 18,347
MI300X 192 GB	Qwen3.5-0.8B	Batched prefill	Pie 98,705 · SGLang 90,331 · vLLM 40,284
MI300X 192 GB	Qwen3-0.6B	Single decode	vLLM 676 · SGLang 541 · Pie 527
MI300X 192 GB	Qwen3-0.6B	Single prefill	Pie 50,202 · SGLang 19,920 · vLLM 19,633
MI300X 192 GB	Qwen3-0.6B	Batched decode	SGLang 32,132 · vLLM 28,108 · Pie 26,936
MI300X 192 GB	Qwen3-0.6B	Batched prefill	Pie 176,715 · SGLang 151,359 · vLLM 66,983
MI300X 192 GB	Gemma-4-E4B	Single decode	Pie 198 · vLLM 194
MI300X 192 GB	Gemma-4-E4B	Single prefill	Pie 15,259 · vLLM 11,773
MI300X 192 GB	Gemma-4-E4B	Batched decode	Pie 11,449 · vLLM 10,608 · SGLang 9,901
MI300X 192 GB	Gemma-4-E4B	Batched prefill	Pie 43,380 · vLLM 32,117 · SGLang 9,008
MI300X 192 GB	Qwen3.6-27B	Single decode	Pie 67.9 · vLLM 64.7 · SGLang 53.9
MI300X 192 GB	Qwen3.6-27B	Single prefill	SGLang 5,523 · vLLM 5,310 · Pie 4,007
MI300X 192 GB	Qwen3.6-27B	Batched decode	vLLM 2,768 · SGLang 2,727
MI300X 192 GB	Qwen3.6-27B	Batched prefill	SGLang 8,708 · vLLM 8,381
MI300X 192 GB	GPT-OSS-20B	Single decode	Pie 304 · vLLM 292
MI300X 192 GB	GPT-OSS-20B	Single prefill	Pie 17,080 · vLLM 10,868
MI300X 192 GB	GPT-OSS-20B	Batched decode	Pie 9,914 · vLLM 7,061
MI300X 192 GB	GPT-OSS-20B	Batched prefill	Pie 49,237 · vLLM 19,115
MI300X 192 GB	GPT-OSS-120B	Single decode	Pie 206 · vLLM 202
MI300X 192 GB	GPT-OSS-120B	Single prefill	Pie 10,679 · vLLM 5,733
MI300X 192 GB	Qwen3.6-35B	Single decode	Pie 239 · SGLang 216 · vLLM 159
MI300X 192 GB	Qwen3.6-35B	Single prefill	Pie 12,837 · SGLang 6,959 · vLLM 5,699
MI300X 192 GB	Qwen3.6-35B	Batched decode	Pie 7,587 · SGLang 7,464 · vLLM 6,308
MI300X 192 GB	Qwen3.6-35B	Batched prefill	Pie 38,645 · SGLang 23,035 · vLLM 19,437
MI300X 192 GB	Muse-Glimmer-30B	Single decode	Pie 63.9 · vLLM 61.1 · SGLang 51.8
MI300X 192 GB	Muse-Glimmer-30B	Single prefill	vLLM 5,907 · SGLang 5,871 · Pie 4,703
MI300X 192 GB	Muse-Glimmer-30B	Batched decode	Pie 3,853
MI300X 192 GB	Muse-Glimmer-30B	Batched prefill	vLLM 9,048 · Pie 9,023
MI300X 192 GB	Gemma-4-26B	Single decode	Pie 180 · vLLM 179 · SGLang 144
MI300X 192 GB	Gemma-4-26B	Single prefill	Pie 14,101 · vLLM 10,968 · SGLang 9,872
MI300X 192 GB	Gemma-4-26B	Batched decode	vLLM 7,286 · Pie 7,071 · SGLang 6,969
MI300X 192 GB	Gemma-4-26B	Batched prefill	Pie 36,554 · SGLang 14,543 · vLLM 11,497
MI300X 192 GB	Gemma-4-31B	Single decode	Pie 57.7 · vLLM 56.7
MI300X 192 GB	Gemma-4-31B	Single prefill	vLLM 5,229 · Pie 4,094
MI300X 192 GB	Gemma-4-31B	Batched decode	Pie 2,786 · vLLM 2,440
MI300X 192 GB	Gemma-4-31B	Batched prefill	Pie 7,886 · vLLM 7,484
MI300X 192 GB	Qwen3.8-27B	Single decode	Pie 67.8 · vLLM 64.2 · SGLang 52.9
MI300X 192 GB	Qwen3.8-27B	Single prefill	SGLang 5,391 · vLLM 5,134 · Pie 4,002
MI300X 192 GB	Qwen3.8-27B	Batched decode	Pie 3,186 · vLLM 3,087 · SGLang 2,928
MI300X 192 GB	Qwen3.8-27B	Batched prefill	SGLang 9,160 · Pie 8,832 · vLLM 8,013