GPT-OSS benchmarks

Results as tablesRelease main-f03640150, as of 2026-11-13

A snapshot taken when this site was last built. The charts above read live data and may be newer. The same tables as markdown.

Throughput on Apple Silicon, tokens per second: every engine on the same model and machine, fastest first
Machine	Model	Setup	Engines (tok/s)
M5 Max 128 GB	GPT-OSS-20B	Single decode	Pie 184 · llama.cpp 151 · MLX 149
M5 Max 128 GB	GPT-OSS-20B	Single prefill	Pie 6,092 · MLX 5,024 · llama.cpp 3,036
M5 Max 128 GB	GPT-OSS-20B	Batched decode	Pie 648 · llama.cpp 532 · MLX 134
M5 Max 128 GB	GPT-OSS-20B	Batched prefill	Pie 6,425 · MLX 4,939 · llama.cpp 3,726
M5 Max 128 GB	GPT-OSS-120B	Single decode	Pie 122 · llama.cpp 106 · MLX 97.5
M5 Max 128 GB	GPT-OSS-120B	Single prefill	MLX 2,367 · llama.cpp 1,530 · Pie 1,234
M5 Max 128 GB	GPT-OSS-120B	Batched decode	llama.cpp 250 · Pie 227 · MLX 83
M5 Max 128 GB	GPT-OSS-120B	Batched prefill	Pie 2,700 · llama.cpp 1,928
M5 Max 64 GB	GPT-OSS-20B	Single decode	Pie 184 · vLLM Metal 158 · llama.cpp 148 · MLX 148
M5 Max 64 GB	GPT-OSS-20B	Single prefill	Pie 6,052 · MLX 4,911 · vLLM Metal 4,714 · llama.cpp 2,909
M5 Max 64 GB	GPT-OSS-20B	Batched decode	vLLM Metal 736 · Pie 597 · llama.cpp 497 · MLX 129
M5 Max 64 GB	GPT-OSS-20B	Batched prefill	Pie 6,116 · vLLM Metal 6,008 · MLX 4,538 · llama.cpp 3,429
M4 Pro 48 GB	GPT-OSS-20B	Single decode	Pie 97.9 · vLLM Metal 90 · MLX 85 · llama.cpp 80.7
M4 Pro 48 GB	GPT-OSS-20B	Single prefill	Pie 1,028 · MLX 893 · vLLM Metal 862 · llama.cpp 810
M4 Pro 48 GB	GPT-OSS-20B	Batched decode	Pie 250 · vLLM Metal 219 · llama.cpp 193 · MLX 71.7
M4 Pro 48 GB	GPT-OSS-20B	Batched prefill	Pie 1,113 · vLLM Metal 944 · llama.cpp 912 · MLX 897

Throughput on NVIDIA GPU, tokens per second: every engine on the same model and machine, fastest first
Machine	Model	Setup	Engines (tok/s)
B200 180 GB	GPT-OSS-20B	Single decode	SGLang 569 · Pie 550 · vLLM 456 · TensorRT-LLM 285
B200 180 GB	GPT-OSS-20B	Single prefill	Pie 36,203 · SGLang 30,644 · vLLM 27,768 · TensorRT-LLM 7,475
B200 180 GB	GPT-OSS-20B	Batched decode	vLLM 43,895 · SGLang 38,625 · Pie 35,733 · TensorRT-LLM 11,287
B200 180 GB	GPT-OSS-20B	Batched prefill	vLLM 138,139 · Pie 115,227 · SGLang 94,130 · TensorRT-LLM 79,131
B200 180 GB	GPT-OSS-120B	Single decode	TensorRT-LLM 428 · SGLang 418 · vLLM 397 · Pie 378
B200 180 GB	GPT-OSS-120B	Single prefill	Pie 19,318 · vLLM 18,538 · SGLang 18,177 · TensorRT-LLM 13,497
B200 180 GB	GPT-OSS-120B	Batched decode	vLLM 28,699 · SGLang 24,257 · Pie 20,408 · TensorRT-LLM 11,495
B200 180 GB	GPT-OSS-120B	Batched prefill	vLLM 90,339 · Pie 72,754 · SGLang 62,902 · TensorRT-LLM 62,034
RTX PRO 6000 96 GB	GPT-OSS-20B	Single decode	Pie 280 · vLLM 260 · SGLang 245 · TensorRT-LLM 227
RTX PRO 6000 96 GB	GPT-OSS-20B	Single prefill	Pie 24,159 · SGLang 19,338 · vLLM 18,268 · TensorRT-LLM 13,991
RTX PRO 6000 96 GB	GPT-OSS-20B	Batched decode	TensorRT-LLM 23,048 · Pie 19,240 · vLLM 17,141 · SGLang 13,874
RTX PRO 6000 96 GB	GPT-OSS-20B	Batched prefill	TensorRT-LLM 65,736 · Pie 49,242 · vLLM 41,689 · SGLang 31,920
RTX PRO 6000 96 GB	GPT-OSS-120B	Single decode	Pie 206 · vLLM 188 · SGLang 178 · TensorRT-LLM 156
RTX PRO 6000 96 GB	GPT-OSS-120B	Single prefill	Pie 12,849 · vLLM 11,919 · TensorRT-LLM 11,822 · SGLang 10,070
RTX PRO 6000 96 GB	GPT-OSS-120B	Batched decode	TensorRT-LLM 11,517 · vLLM 8,470 · Pie 7,998 · SGLang 3,673
RTX PRO 6000 96 GB	GPT-OSS-120B	Batched prefill	TensorRT-LLM 42,843 · Pie 30,372 · vLLM 27,725 · SGLang 21,200
RTX 5090 32 GB	GPT-OSS-20B	Single decode	Pie 318 · vLLM 293 · SGLang 279 · TensorRT-LLM 233
RTX 5090 32 GB	GPT-OSS-20B	Single prefill	Pie 19,891 · vLLM 16,812 · SGLang 16,244 · TensorRT-LLM 15,240
RTX 5090 32 GB	GPT-OSS-20B	Batched decode	TensorRT-LLM 16,098 · Pie 11,771 · vLLM 10,893 · SGLang 9,979
RTX 5090 32 GB	GPT-OSS-20B	Batched prefill	TensorRT-LLM 52,472 · Pie 32,873 · vLLM 27,370 · SGLang 23,606
RTX PRO 4500 32 GB	GPT-OSS-20B	Single decode	Pie 183 · vLLM 174 · SGLang 170 · TensorRT-LLM 140
RTX PRO 4500 32 GB	GPT-OSS-20B	Single prefill	TensorRT-LLM 14,148 · Pie 13,893 · vLLM 11,703 · SGLang 10,567
RTX PRO 4500 32 GB	GPT-OSS-20B	Batched decode	TensorRT-LLM 8,977 · Pie 7,652 · vLLM 6,851 · SGLang 5,833
RTX PRO 4500 32 GB	GPT-OSS-20B	Batched prefill	TensorRT-LLM 27,230 · Pie 20,379 · vLLM 17,659 · SGLang 13,804
H200 SXM 141 GB	GPT-OSS-20B	Single decode	Pie 438 · vLLM 328 · TensorRT-LLM 324 · SGLang 317
H200 SXM 141 GB	GPT-OSS-20B	Single prefill	vLLM 29,595 · Pie 28,862 · SGLang 25,162 · TensorRT-LLM 10,227
H200 SXM 141 GB	GPT-OSS-20B	Batched decode	Pie 24,792 · SGLang 22,032 · vLLM 22,012 · TensorRT-LLM 20,447
H200 SXM 141 GB	GPT-OSS-20B	Batched prefill	SGLang 66,327 · Pie 60,913 · TensorRT-LLM 60,737 · vLLM 58,424
H200 SXM 141 GB	GPT-OSS-120B	Single decode	Pie 290 · vLLM 225 · TensorRT-LLM 224 · SGLang 216
H200 SXM 141 GB	GPT-OSS-120B	Single prefill	SGLang 13,915 · vLLM 13,488 · Pie 9,874 · TensorRT-LLM 7,125
H200 SXM 141 GB	GPT-OSS-120B	Batched decode	SGLang 12,529 · TensorRT-LLM 11,346 · vLLM 10,981 · Pie 9,688
H200 SXM 141 GB	GPT-OSS-120B	Batched prefill	vLLM 41,652 · SGLang 36,554 · TensorRT-LLM 33,697 · Pie 31,120
H100 SXM 80 GB	GPT-OSS-20B	Single decode	Pie 385 · vLLM 305 · TensorRT-LLM 299 · SGLang 298
H100 SXM 80 GB	GPT-OSS-20B	Single prefill	Pie 22,779 · vLLM 22,379 · SGLang 20,203 · TensorRT-LLM 10,169
H100 SXM 80 GB	GPT-OSS-20B	Batched decode	SGLang 21,360 · vLLM 18,398 · Pie 15,291 · TensorRT-LLM 10,500
H100 SXM 80 GB	GPT-OSS-20B	Batched prefill	vLLM 66,370 · SGLang 63,978 · Pie 59,779 · TensorRT-LLM 58,560
H100 SXM 80 GB	GPT-OSS-120B	Single decode	Pie 263 · vLLM 214 · SGLang 204 · TensorRT-LLM 203
H100 SXM 80 GB	GPT-OSS-120B	Single prefill	SGLang 12,998 · vLLM 11,811 · Pie 10,177 · TensorRT-LLM 6,216
H100 SXM 80 GB	GPT-OSS-120B	Batched decode	TensorRT-LLM 7,349 · vLLM 6,290 · Pie 5,225 · SGLang 3,683
H100 SXM 80 GB	GPT-OSS-120B	Batched prefill	SGLang 37,544 · TensorRT-LLM 34,291 · Pie 31,886 · vLLM 25,417
L40S 46 GB	GPT-OSS-20B	Single decode	Pie 162 · vLLM 158 · SGLang 137
L40S 46 GB	GPT-OSS-20B	Single prefill	Pie 18,678 · vLLM 16,734 · SGLang 639
L40S 46 GB	GPT-OSS-20B	Batched prefill	Pie 27,648 · vLLM 23,099 · SGLang 1,066
RTX 4090 24 GB	GPT-OSS-20B	Single decode	Pie 182 · vLLM 176 · SGLang 150
RTX 4090 24 GB	GPT-OSS-20B	Single prefill	Pie 14,393 · vLLM 12,569 · SGLang 806
RTX 4090 24 GB	GPT-OSS-20B	Batched prefill	Pie 20,423 · vLLM 19,483 · SGLang 1,529
A100 SXM 80 GB	GPT-OSS-20B	Single decode	Pie 257 · vLLM 238 · SGLang 217
A100 SXM 80 GB	GPT-OSS-20B	Single prefill	Pie 15,977 · vLLM 13,818 · SGLang 11,657
A100 SXM 80 GB	GPT-OSS-20B	Batched decode	Pie 9,277 · vLLM 8,807 · SGLang 7,450
A100 SXM 80 GB	GPT-OSS-20B	Batched prefill	Pie 24,086 · vLLM 20,222 · SGLang 16,265
A100 SXM 80 GB	GPT-OSS-120B	Single decode	Pie 182 · vLLM 166 · SGLang 150
A100 SXM 80 GB	GPT-OSS-120B	Single prefill	Pie 6,972 · vLLM 6,930 · SGLang 6,104
A100 SXM 80 GB	GPT-OSS-120B	Batched decode	vLLM 2,598 · Pie 2,177 · SGLang 1,117
A100 SXM 80 GB	GPT-OSS-120B	Batched prefill	Pie 12,262 · vLLM 10,017 · SGLang 9,320
A40 48 GB	GPT-OSS-20B	Single decode	Pie 129 · vLLM 123 · SGLang 120
A40 48 GB	GPT-OSS-20B	Single prefill	Pie 7,941 · vLLM 7,625 · SGLang 6,390
A40 48 GB	GPT-OSS-20B	Batched decode	vLLM 4,477 · Pie 3,761 · SGLang 3,330
A40 48 GB	GPT-OSS-20B	Batched prefill	Pie 11,893 · vLLM 11,573 · SGLang 7,937

Throughput on AMD GPU, tokens per second: every engine on the same model and machine, fastest first
Machine	Model	Setup	Engines (tok/s)
MI355X 288 GB	GPT-OSS-20B	Single decode	SGLang 444 · vLLM 366 · Pie 358
MI355X 288 GB	GPT-OSS-20B	Single prefill	vLLM 51,812 · Pie 30,269 · SGLang 18,973
MI355X 288 GB	GPT-OSS-20B	Batched decode	SGLang 43,649 · vLLM 30,460 · Pie 22,617
MI355X 288 GB	GPT-OSS-20B	Batched prefill	SGLang 133,334 · vLLM 80,988 · Pie 68,276
MI355X 288 GB	GPT-OSS-120B	Single decode	SGLang 321 · vLLM 247 · Pie 245
MI355X 288 GB	GPT-OSS-120B	Single prefill	vLLM 30,518 · Pie 17,185 · SGLang 11,863
MI355X 288 GB	GPT-OSS-120B	Batched decode	SGLang 30,132 · vLLM 21,092 · Pie 13,619
MI355X 288 GB	GPT-OSS-120B	Batched prefill	SGLang 116,011 · vLLM 53,707 · Pie 42,272
MI350X 288 GB	GPT-OSS-20B	Single decode	SGLang 422 · vLLM 350 · Pie 341
MI350X 288 GB	GPT-OSS-20B	Single prefill	vLLM 46,013 · Pie 27,206 · SGLang 17,014
MI350X 288 GB	GPT-OSS-120B	Single decode	SGLang 306 · vLLM 236 · Pie 233
MI350X 288 GB	GPT-OSS-120B	Single prefill	vLLM 25,435 · Pie 15,228 · SGLang 11,543
MI300X 192 GB	GPT-OSS-20B	Single decode	Pie 304 · vLLM 292
MI300X 192 GB	GPT-OSS-20B	Single prefill	Pie 17,080 · vLLM 10,868
MI300X 192 GB	GPT-OSS-20B	Batched decode	Pie 9,914 · vLLM 7,061
MI300X 192 GB	GPT-OSS-20B	Batched prefill	Pie 49,237 · vLLM 19,115
MI300X 192 GB	GPT-OSS-120B	Single decode	Pie 206 · vLLM 202
MI300X 192 GB	GPT-OSS-120B	Single prefill	Pie 10,679 · vLLM 5,733

Throughput on Google TPU, tokens per second: every engine on the same model and machine, fastest first
Machine	Model	Setup	Engines (tok/s)
TPU v6e 32 GB	GPT-OSS-20B	Single decode	Pie 216 · vLLM 110
TPU v6e 32 GB	GPT-OSS-20B	Single prefill	Pie 20,938 · vLLM 15,629
TPU v6e 32 GB	GPT-OSS-20B	Batched decode	Pie 6,710 · vLLM 5,960
TPU v6e 32 GB	GPT-OSS-20B	Batched prefill	Pie 36,384 · vLLM 33,880