Google TPU benchmarks

Results as tablesRelease main-f03640150, as of 2026-11-13

A snapshot taken when this site was last built. The charts above read live data and may be newer. The same tables as markdown.

Throughput on Google TPU, tokens per second: every engine on the same model and machine, fastest first
Machine	Model	Setup	Engines (tok/s)
TPU v6e 32 GB	Qwen3.5-0.8B	Single decode	Pie 688 · vLLM 348 · SGLang 214
TPU v6e 32 GB	Qwen3.5-0.8B	Single prefill	vLLM 49,984 · Pie 49,751 · SGLang 28,941
TPU v6e 32 GB	Qwen3.5-0.8B	Batched decode	vLLM 16,189 · Pie 14,635 · SGLang 5,464
TPU v6e 32 GB	Qwen3.5-0.8B	Batched prefill	vLLM 140,661 · Pie 116,810 · SGLang 116,589
TPU v6e 32 GB	Qwen3-0.6B	Single decode	Pie 694 · vLLM 421 · SGLang 329
TPU v6e 32 GB	Qwen3-0.6B	Single prefill	vLLM 61,879 · Pie 58,756 · SGLang 38,262
TPU v6e 32 GB	Qwen3-0.6B	Batched decode	vLLM 12,086 · Pie 11,620 · SGLang 7,225
TPU v6e 32 GB	Qwen3-0.6B	Batched prefill	Pie 236,161 · vLLM 230,690 · SGLang 112,732
TPU v6e 32 GB	Gemma-4-E2B	Single decode	vLLM 235 · Pie 231
TPU v6e 32 GB	Gemma-4-E2B	Single prefill	Pie 49,315 · vLLM 36,201
TPU v6e 32 GB	Gemma-4-E2B	Batched decode	vLLM 15,113 · Pie 13,966
TPU v6e 32 GB	Gemma-4-E2B	Batched prefill	vLLM 115,393 · Pie 99,137
TPU v6e 32 GB	Gemma-4-E4B	Single decode	vLLM 117 · Pie 116
TPU v6e 32 GB	Gemma-4-E4B	Single prefill	Pie 34,947 · vLLM 23,540
TPU v6e 32 GB	Gemma-4-E4B	Batched decode	vLLM 8,605 · Pie 8,359
TPU v6e 32 GB	Gemma-4-E4B	Batched prefill	vLLM 65,764 · Pie 61,438
TPU v6e 32 GB	Gemma-4-12B	Single decode	vLLM 45.4 · Pie 45.3
TPU v6e 32 GB	Gemma-4-12B	Single prefill	Pie 13,725 · vLLM 11,182
TPU v6e 32 GB	Gemma-4-12B	Batched decode	vLLM 1,195 · Pie 1,129
TPU v6e 32 GB	Gemma-4-12B	Batched prefill	vLLM 23,773 · Pie 21,796
TPU v6e 32 GB	GPT-OSS-20B	Single decode	Pie 216 · vLLM 110
TPU v6e 32 GB	GPT-OSS-20B	Single prefill	Pie 20,938 · vLLM 15,629
TPU v6e 32 GB	GPT-OSS-20B	Batched decode	Pie 6,710 · vLLM 5,960
TPU v6e 32 GB	GPT-OSS-20B	Batched prefill	Pie 36,384 · vLLM 33,880
TPU v5e 16 GB	Qwen3.5-0.8B	Single decode	Pie 467 · vLLM 271 · SGLang 175
TPU v5e 16 GB	Qwen3.5-0.8B	Single prefill	Pie 43,296 · vLLM 32,315 · SGLang 6,798
TPU v5e 16 GB	Qwen3.5-0.8B	Batched decode	vLLM 8,759 · Pie 8,298 · SGLang 3,001
TPU v5e 16 GB	Qwen3.5-0.8B	Batched prefill	vLLM 73,279 · SGLang 60,116 · Pie 54,344
TPU v5e 16 GB	Qwen3-0.6B	Single decode	Pie 555 · vLLM 326 · SGLang 264
TPU v5e 16 GB	Qwen3-0.6B	Single prefill	Pie 61,458 · vLLM 38,439 · SGLang 26,055
TPU v5e 16 GB	Qwen3-0.6B	Batched decode	vLLM 9,028 · Pie 8,124 · SGLang 5,240
TPU v5e 16 GB	Qwen3-0.6B	Batched prefill	vLLM 109,487 · Pie 106,037 · SGLang 73,324
TPU v5e 16 GB	Gemma-4-E2B	Single decode	Pie 152 · vLLM 148
TPU v5e 16 GB	Gemma-4-E2B	Single prefill	Pie 26,042 · vLLM 17,095
TPU v5e 16 GB	Gemma-4-E2B	Batched decode	vLLM 8,811 · Pie 6,891
TPU v5e 16 GB	Gemma-4-E2B	Batched prefill	vLLM 39,846 · Pie 29,543
TPU v5e 16 GB	Gemma-4-E4B	Single decode	vLLM 74.1 · Pie 68.2
TPU v5e 16 GB	Gemma-4-E4B	Single prefill	Pie 14,526 · vLLM 9,345
TPU v5e 16 GB	Gemma-4-E4B	Batched decode	vLLM 1,370 · Pie 1,309
TPU v5e 16 GB	Gemma-4-E4B	Batched prefill	vLLM 20,053 · Pie 15,908