Correctness
Output parity against reference engines on matched weights.
Results as tablesRelease main-f03640150, as of 2026-11-13
A snapshot taken when this site was last built. The charts above read live data and may be newer. The same tables as markdown.
| Platform | Machine | Model | Result | Check |
|---|---|---|---|---|
| Apple Silicon | M5 Max 128 GB | Muse-Spark-Open-816B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Apple Silicon | M5 Max 128 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Apple Silicon | M5 Max 64 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Apple Silicon | M5 Max 64 GB | Muse-Spark-Open-816B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Apple Silicon | M5 Max 64 GB | GPT-OSS-120B | Pass | Next-token KL divergence vs MLX: 0.0001 nats |
| Apple Silicon | M5 Max 64 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Apple Silicon | M4 Pro 48 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Apple Silicon | M4 Pro 48 GB | Muse-Spark-Open-816B | Pass | Next-token KL divergence vs MLX: 0.0001 nats |
| Apple Silicon | M4 Pro 48 GB | GPT-OSS-120B | Pass | Next-token KL divergence vs MLX: 0.0005 nats |
| Apple Silicon | M4 Pro 48 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs MLX: 0.0001 nats |
| Apple Silicon | M3 16 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Apple Silicon | M3 16 GB | Qwen3.8-27B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Apple Silicon | M3 16 GB | Qwen3.6-35B | Pass | Next-token KL divergence vs MLX: 0.0006 nats |
| Apple Silicon | M3 16 GB | Qwen3.6-27B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Apple Silicon | M3 16 GB | Muse-Spark-Open-816B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Apple Silicon | M3 16 GB | Muse-Glimmer-30B | Pass | Next-token KL divergence vs MLX: 0.0001 nats |
| Apple Silicon | M3 16 GB | Gemma-4-31B | Pass | Next-token KL divergence vs MLX: 0.0175 nats |
| Apple Silicon | M3 16 GB | Gemma-4-26B | Pass | Next-token KL divergence vs MLX: 0.1318 nats |
| Apple Silicon | M3 16 GB | GPT-OSS-120B | Pass | Next-token KL divergence vs MLX: 0.0015 nats |
| Apple Silicon | M3 16 GB | GPT-OSS-20B | Pass | Next-token KL divergence vs MLX: 0.0001 nats |
| Apple Silicon | M3 16 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Apple Silicon | M2 16 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Apple Silicon | M2 16 GB | Qwen3.8-27B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Apple Silicon | M2 16 GB | Qwen3.6-35B | Pass | Next-token KL divergence vs MLX: 0.0006 nats |
| Apple Silicon | M2 16 GB | Qwen3.6-27B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Apple Silicon | M2 16 GB | Muse-Spark-Open-816B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Apple Silicon | M2 16 GB | Muse-Glimmer-30B | Pass | Next-token KL divergence vs MLX: 0.0001 nats |
| Apple Silicon | M2 16 GB | Gemma-4-31B | Pass | Next-token KL divergence vs MLX: 0.0062 nats |
| Apple Silicon | M2 16 GB | Gemma-4-26B | Pass | Next-token KL divergence vs MLX: 0.0390 nats |
| Apple Silicon | M2 16 GB | GPT-OSS-120B | Pass | Next-token KL divergence vs MLX: 0.0019 nats |
| Apple Silicon | M2 16 GB | GPT-OSS-20B | Pass | Next-token KL divergence vs MLX: 0.0002 nats |
| Apple Silicon | M2 16 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| NVIDIA GPU | B300 288 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | B300 288 GB | Kimi-K3-2.8T | Pass | Next-token KL divergence vs vLLM: 0.0005 nats |
| NVIDIA GPU | B300 288 GB | GLM-5.3-Flash-320B | Pass | Next-token KL divergence vs vLLM: 0.0007 nats |
| NVIDIA GPU | B300 288 GB | MiMo-V2.6-Flash-RL-309B | Pass | Next-token KL divergence vs vLLM: 0.0009 nats |
| NVIDIA GPU | B300 288 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | B200 180 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | B200 180 GB | Kimi-K3-2.8T | Pass | Next-token KL divergence vs vLLM: 0.0004 nats |
| NVIDIA GPU | B200 180 GB | DeepSeek-V4.1-Flash-552B | Pass | Next-token KL divergence vs vLLM: 0.0497 nats |
| NVIDIA GPU | B200 180 GB | DeepSeek-V4-Flash-284B | Pass | Next-token KL divergence vs vLLM: 0.0355 nats |
| NVIDIA GPU | B200 180 GB | Muse-Spark-Open-816B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| NVIDIA GPU | B200 180 GB | GLM-5.3-744B | Pass | Next-token KL divergence vs vLLM: 0.0211 nats |
| NVIDIA GPU | B200 180 GB | GLM-5.3-Flash-320B | Pass | Next-token KL divergence vs vLLM: 0.0005 nats |
| NVIDIA GPU | B200 180 GB | MiniMax-M3.1-Flash-428B | Pass | Next-token KL divergence vs vLLM: 0.0168 nats |
| NVIDIA GPU | B200 180 GB | MiMo-V2.6-Flash-RL-309B | Pass | Next-token KL divergence vs vLLM: 0.0004 nats |
| NVIDIA GPU | B200 180 GB | Hy4-770B | Pass | Next-token KL divergence vs vLLM: 0.0012 nats |
| NVIDIA GPU | B200 180 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | B200 180 GB | Nemotron-3-Super-120B | Pass | Next-token KL divergence vs vLLM: 0.0044 nats |
| NVIDIA GPU | RTX PRO 6000 96 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | RTX PRO 6000 96 GB | Kimi-K3-2.8T | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | RTX PRO 6000 96 GB | DeepSeek-V4.1-Flash-552B | Pass | Next-token KL divergence vs vLLM: 0.0339 nats |
| NVIDIA GPU | RTX PRO 6000 96 GB | DeepSeek-V4-Flash-284B | Pass | Next-token KL divergence vs vLLM: 0.0122 nats |
| NVIDIA GPU | RTX PRO 6000 96 GB | Muse-Spark-Open-816B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | RTX PRO 6000 96 GB | GLM-5.3-744B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | RTX PRO 6000 96 GB | GLM-5.3-Flash-320B | Pass | Next-token KL divergence vs vLLM: 0.0002 nats |
| NVIDIA GPU | RTX PRO 6000 96 GB | MiniMax-M3.1-Flash-428B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| NVIDIA GPU | RTX PRO 6000 96 GB | MiMo-V2.6-Flash-RL-309B | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | RTX PRO 6000 96 GB | Hy4-770B | Pass | Next-token KL divergence vs vLLM: 0.0223 nats |
| NVIDIA GPU | RTX PRO 6000 96 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | RTX PRO 6000 96 GB | Nemotron-3-Super-120B | Pass | Next-token KL divergence vs vLLM: 0.0002 nats |
| NVIDIA GPU | RTX 5090 32 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | RTX 5090 32 GB | Qwen3.8-27B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | RTX 5090 32 GB | Qwen3.6-35B | Pass | Next-token KL divergence vs vLLM: 0.0007 nats |
| NVIDIA GPU | RTX 5090 32 GB | Qwen3.6-27B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | RTX 5090 32 GB | Kimi-K3-2.8T | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | RTX 5090 32 GB | DeepSeek-V4.1-Flash-552B | Pass | Next-token KL divergence vs vLLM: 0.0362 nats |
| NVIDIA GPU | RTX 5090 32 GB | DeepSeek-V4-Flash-284B | Pass | Next-token KL divergence vs vLLM: 0.0090 nats |
| NVIDIA GPU | RTX 5090 32 GB | Muse-Spark-Open-816B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| NVIDIA GPU | RTX 5090 32 GB | Muse-Glimmer-30B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| NVIDIA GPU | RTX 5090 32 GB | Gemma-4-26B | Pass | Next-token KL divergence vs vLLM: 0.1493 nats |
| NVIDIA GPU | RTX 5090 32 GB | GPT-OSS-120B | Pass | Next-token KL divergence vs vLLM: 0.0290 nats |
| NVIDIA GPU | RTX 5090 32 GB | GLM-5.3-744B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| NVIDIA GPU | RTX 5090 32 GB | GLM-5.3-Flash-320B | Pass | Next-token KL divergence vs vLLM: 0.0004 nats |
| NVIDIA GPU | RTX 5090 32 GB | MiniMax-M3.1-Flash-428B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| NVIDIA GPU | RTX 5090 32 GB | MiMo-V2.6-Flash-RL-309B | Pass | Next-token KL divergence vs vLLM: 0.0004 nats |
| NVIDIA GPU | RTX 5090 32 GB | Hy4-770B | Pass | Next-token KL divergence vs vLLM: 0.0239 nats |
| NVIDIA GPU | RTX 5090 32 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | RTX 5090 32 GB | Nemotron-3-Super-120B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| NVIDIA GPU | RTX PRO 4500 32 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | RTX PRO 4500 32 GB | Qwen3.8-27B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | RTX PRO 4500 32 GB | Qwen3.6-35B | Pass | Next-token KL divergence vs vLLM: 0.0005 nats |
| NVIDIA GPU | RTX PRO 4500 32 GB | Qwen3.6-27B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | RTX PRO 4500 32 GB | Kimi-K3-2.8T | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | RTX PRO 4500 32 GB | DeepSeek-V4.1-Flash-552B | Pass | Next-token KL divergence vs vLLM: 0.0310 nats |
| NVIDIA GPU | RTX PRO 4500 32 GB | DeepSeek-V4-Flash-284B | Pass | Next-token KL divergence vs vLLM: 0.0076 nats |
| NVIDIA GPU | RTX PRO 4500 32 GB | Muse-Spark-Open-816B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| NVIDIA GPU | RTX PRO 4500 32 GB | Muse-Glimmer-30B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| NVIDIA GPU | RTX PRO 4500 32 GB | Gemma-4-26B | Pass | Next-token KL divergence vs vLLM: 0.0301 nats |
| NVIDIA GPU | RTX PRO 4500 32 GB | GPT-OSS-120B | Pass | Next-token KL divergence vs vLLM: 0.0262 nats |
| NVIDIA GPU | RTX PRO 4500 32 GB | GLM-5.3-744B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | RTX PRO 4500 32 GB | GLM-5.3-Flash-320B | Pass | Next-token KL divergence vs vLLM: 0.0002 nats |
| NVIDIA GPU | RTX PRO 4500 32 GB | MiniMax-M3.1-Flash-428B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| NVIDIA GPU | RTX PRO 4500 32 GB | MiMo-V2.6-Flash-RL-309B | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | RTX PRO 4500 32 GB | Hy4-770B | Pass | Next-token KL divergence vs vLLM: 0.0228 nats |
| NVIDIA GPU | RTX PRO 4500 32 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | RTX PRO 4500 32 GB | Nemotron-3-Super-120B | Pass | Next-token KL divergence vs vLLM: 0.0002 nats |
| NVIDIA GPU | H200 SXM 141 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | H200 SXM 141 GB | Kimi-K3-2.8T | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | H200 SXM 141 GB | DeepSeek-V4.1-Flash-552B | Pass | Next-token KL divergence vs vLLM: 0.0101 nats |
| NVIDIA GPU | H200 SXM 141 GB | DeepSeek-V4-Flash-284B | Pass | Next-token KL divergence vs vLLM: 0.0136 nats |
| NVIDIA GPU | H200 SXM 141 GB | Muse-Spark-Open-816B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| NVIDIA GPU | H200 SXM 141 GB | GLM-5.3-744B | Pass | Next-token KL divergence vs vLLM: 0.0210 nats |
| NVIDIA GPU | H200 SXM 141 GB | GLM-5.3-Flash-320B | Pass | Next-token KL divergence vs vLLM: 0.0004 nats |
| NVIDIA GPU | H200 SXM 141 GB | MiniMax-M3.1-Flash-428B | Pass | Next-token KL divergence vs vLLM: 0.0452 nats |
| NVIDIA GPU | H200 SXM 141 GB | MiMo-V2.6-Flash-RL-309B | Pass | Next-token KL divergence vs vLLM: 0.0147 nats |
| NVIDIA GPU | H200 SXM 141 GB | Hy4-770B | Pass | Next-token KL divergence vs vLLM: 0.0011 nats |
| NVIDIA GPU | H200 SXM 141 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | H200 SXM 141 GB | Nemotron-3-Super-120B | Pass | Next-token KL divergence vs vLLM: 0.0049 nats |
| NVIDIA GPU | H100 SXM 80 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | H100 SXM 80 GB | Qwen3.6-35B | Pass | Next-token KL divergence vs vLLM: 0.0005 nats |
| NVIDIA GPU | H100 SXM 80 GB | Kimi-K3-2.8T | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | H100 SXM 80 GB | DeepSeek-V4.1-Flash-552B | Pass | Next-token KL divergence vs vLLM: 0.0170 nats |
| NVIDIA GPU | H100 SXM 80 GB | DeepSeek-V4-Flash-284B | Pass | Next-token KL divergence vs vLLM: 0.0078 nats |
| NVIDIA GPU | H100 SXM 80 GB | Muse-Spark-Open-816B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | H100 SXM 80 GB | GLM-5.3-744B | Pass | Next-token KL divergence vs vLLM: 0.0165 nats |
| NVIDIA GPU | H100 SXM 80 GB | GLM-5.3-Flash-320B | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | H100 SXM 80 GB | MiniMax-M3.1-Flash-428B | Pass | Next-token KL divergence vs vLLM: 0.0059 nats |
| NVIDIA GPU | H100 SXM 80 GB | MiMo-V2.6-Flash-RL-309B | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | H100 SXM 80 GB | Hy4-770B | Pass | Next-token KL divergence vs vLLM: 0.0010 nats |
| NVIDIA GPU | H100 SXM 80 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | H100 SXM 80 GB | Nemotron-3-Super-120B | Pass | Next-token KL divergence vs vLLM: 0.0010 nats |
| NVIDIA GPU | L40S 46 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | L40S 46 GB | Qwen3.8-27B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | L40S 46 GB | Qwen3.6-35B | Pass | Next-token KL divergence vs vLLM: 0.0007 nats |
| NVIDIA GPU | L40S 46 GB | Qwen3.6-27B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | L40S 46 GB | Kimi-K3-2.8T | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | L40S 46 GB | DeepSeek-V4.1-Flash-552B | Pass | Next-token KL divergence vs vLLM: 0.0163 nats |
| NVIDIA GPU | L40S 46 GB | DeepSeek-V4-Flash-284B | Pass | Next-token KL divergence vs vLLM: 0.0069 nats |
| NVIDIA GPU | L40S 46 GB | Muse-Spark-Open-816B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | L40S 46 GB | Muse-Glimmer-30B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| NVIDIA GPU | L40S 46 GB | Gemma-4-31B | Pass | Next-token KL divergence vs vLLM: 0.0017 nats |
| NVIDIA GPU | L40S 46 GB | Gemma-4-26B | Pass | Next-token KL divergence vs vLLM: 0.0732 nats |
| NVIDIA GPU | L40S 46 GB | GPT-OSS-120B | Pass | Next-token KL divergence vs vLLM: 0.0251 nats |
| NVIDIA GPU | L40S 46 GB | GLM-5.3-744B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | L40S 46 GB | GLM-5.3-Flash-320B | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | L40S 46 GB | MiniMax-M3.1-Flash-428B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | L40S 46 GB | MiMo-V2.6-Flash-RL-309B | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | L40S 46 GB | Hy4-770B | Pass | Next-token KL divergence vs vLLM: 0.0221 nats |
| NVIDIA GPU | L40S 46 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | L40S 46 GB | Nemotron-3-Super-120B | Pass | Next-token KL divergence vs vLLM: 0.0002 nats |
| NVIDIA GPU | RTX 4090 24 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | RTX 4090 24 GB | Qwen3.8-27B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | RTX 4090 24 GB | Qwen3.6-35B | Pass | Next-token KL divergence vs vLLM: 0.0004 nats |
| NVIDIA GPU | RTX 4090 24 GB | Qwen3.6-27B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | RTX 4090 24 GB | Kimi-K3-2.8T | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | RTX 4090 24 GB | DeepSeek-V4.1-Flash-552B | Pass | Next-token KL divergence vs vLLM: 0.0092 nats |
| NVIDIA GPU | RTX 4090 24 GB | DeepSeek-V4-Flash-284B | Pass | Next-token KL divergence vs vLLM: 0.0068 nats |
| NVIDIA GPU | RTX 4090 24 GB | Muse-Spark-Open-816B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| NVIDIA GPU | RTX 4090 24 GB | Muse-Glimmer-30B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| NVIDIA GPU | RTX 4090 24 GB | Gemma-4-26B | Pass | Next-token KL divergence vs vLLM: 0.0620 nats |
| NVIDIA GPU | RTX 4090 24 GB | GPT-OSS-120B | Pass | Next-token KL divergence vs vLLM: 0.0210 nats |
| NVIDIA GPU | RTX 4090 24 GB | GLM-5.3-744B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| NVIDIA GPU | RTX 4090 24 GB | GLM-5.3-Flash-320B | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | RTX 4090 24 GB | MiniMax-M3.1-Flash-428B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | RTX 4090 24 GB | MiMo-V2.6-Flash-RL-309B | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | RTX 4090 24 GB | Hy4-770B | Pass | Next-token KL divergence vs vLLM: 0.0231 nats |
| NVIDIA GPU | RTX 4090 24 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | RTX 4090 24 GB | Nemotron-3-Super-120B | Pass | Next-token KL divergence vs vLLM: 0.0002 nats |
| NVIDIA GPU | A100 SXM 80 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | A100 SXM 80 GB | Qwen3.6-35B | Pass | Next-token KL divergence vs vLLM: 0.0006 nats |
| NVIDIA GPU | A100 SXM 80 GB | Kimi-K3-2.8T | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | A100 SXM 80 GB | DeepSeek-V4.1-Flash-552B | Pass | Next-token KL divergence vs vLLM: 0.0111 nats |
| NVIDIA GPU | A100 SXM 80 GB | DeepSeek-V4-Flash-284B | Pass | Next-token KL divergence vs vLLM: 0.0062 nats |
| NVIDIA GPU | A100 SXM 80 GB | Muse-Spark-Open-816B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| NVIDIA GPU | A100 SXM 80 GB | GLM-5.3-744B | Pass | Next-token KL divergence vs vLLM: 0.0091 nats |
| NVIDIA GPU | A100 SXM 80 GB | GLM-5.3-Flash-320B | Pass | Next-token KL divergence vs vLLM: 0.0002 nats |
| NVIDIA GPU | A100 SXM 80 GB | MiniMax-M3.1-Flash-428B | Pass | Next-token KL divergence vs vLLM: 0.0057 nats |
| NVIDIA GPU | A100 SXM 80 GB | MiMo-V2.6-Flash-RL-309B | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | A100 SXM 80 GB | Hy4-770B | Pass | Next-token KL divergence vs vLLM: 0.0238 nats |
| NVIDIA GPU | A100 SXM 80 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | A100 SXM 80 GB | Nemotron-3-Super-120B | Pass | Next-token KL divergence vs vLLM: 0.0012 nats |
| NVIDIA GPU | A40 48 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | A40 48 GB | Qwen3.8-27B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | A40 48 GB | Qwen3.6-35B | Pass | Next-token KL divergence vs vLLM: 0.0007 nats |
| NVIDIA GPU | A40 48 GB | Qwen3.6-27B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | A40 48 GB | Kimi-K3-2.8T | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | A40 48 GB | DeepSeek-V4.1-Flash-552B | Pass | Next-token KL divergence vs vLLM: 0.0098 nats |
| NVIDIA GPU | A40 48 GB | DeepSeek-V4-Flash-284B | Pass | Next-token KL divergence vs vLLM: 0.0061 nats |
| NVIDIA GPU | A40 48 GB | Muse-Spark-Open-816B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| NVIDIA GPU | A40 48 GB | Muse-Glimmer-30B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| NVIDIA GPU | A40 48 GB | Gemma-4-31B | Pass | Next-token KL divergence vs vLLM: 0.0017 nats |
| NVIDIA GPU | A40 48 GB | Gemma-4-26B | Pass | Next-token KL divergence vs vLLM: 0.0896 nats |
| NVIDIA GPU | A40 48 GB | GPT-OSS-120B | Pass | Next-token KL divergence vs vLLM: 0.0243 nats |
| NVIDIA GPU | A40 48 GB | GLM-5.3-744B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | A40 48 GB | GLM-5.3-Flash-320B | Pass | Next-token KL divergence vs vLLM: 0.0002 nats |
| NVIDIA GPU | A40 48 GB | MiniMax-M3.1-Flash-428B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | A40 48 GB | MiMo-V2.6-Flash-RL-309B | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| NVIDIA GPU | A40 48 GB | Hy4-770B | Pass | Next-token KL divergence vs vLLM: 0.0219 nats |
| NVIDIA GPU | A40 48 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| NVIDIA GPU | A40 48 GB | Nemotron-3-Super-120B | Pass | Next-token KL divergence vs vLLM: 0.0002 nats |
| AMD GPU | MI355X 288 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| AMD GPU | MI355X 288 GB | Kimi-K3-2.8T | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| AMD GPU | MI355X 288 GB | DeepSeek-V4.1-Flash-552B | Pass | Next-token KL divergence vs vLLM: 0.0425 nats |
| AMD GPU | MI355X 288 GB | DeepSeek-V4-Flash-284B | Fail | Next-token KL divergence vs vLLM: 0.7213 nats |
| AMD GPU | MI355X 288 GB | GLM-5.3-744B | Pass | Next-token KL divergence vs vLLM: 0.0123 nats |
| AMD GPU | MI355X 288 GB | GLM-5.3-Flash-320B | Pass | Next-token KL divergence vs vLLM: 0.0020 nats |
| AMD GPU | MI355X 288 GB | MiniMax-M3.1-Flash-428B | Pass | Next-token KL divergence vs vLLM: 0.0075 nats |
| AMD GPU | MI355X 288 GB | MiMo-V2.6-Flash-RL-309B | Pass | Next-token KL divergence vs vLLM: 0.0116 nats |
| AMD GPU | MI355X 288 GB | Hy4-770B | Pass | Next-token KL divergence vs vLLM: 0.0031 nats |
| AMD GPU | MI355X 288 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| AMD GPU | MI355X 288 GB | Nemotron-3-Super-120B | Pass | Next-token KL divergence vs vLLM: 0.0013 nats |
| AMD GPU | MI350X 288 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| AMD GPU | MI350X 288 GB | Kimi-K3-2.8T | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| AMD GPU | MI350X 288 GB | DeepSeek-V4.1-Flash-552B | Pass | Next-token KL divergence vs vLLM: 0.0327 nats |
| AMD GPU | MI350X 288 GB | DeepSeek-V4-Flash-284B | Fail | Next-token KL divergence vs vLLM: 0.4064 nats |
| AMD GPU | MI350X 288 GB | GLM-5.3-744B | Pass | Next-token KL divergence vs vLLM: 0.0131 nats |
| AMD GPU | MI350X 288 GB | GLM-5.3-Flash-320B | Pass | Next-token KL divergence vs vLLM: 0.0019 nats |
| AMD GPU | MI350X 288 GB | MiMo-V2.6-Flash-RL-309B | Pass | Next-token KL divergence vs vLLM: 0.0120 nats |
| AMD GPU | MI350X 288 GB | Hy4-770B | Pass | Next-token KL divergence vs vLLM: 0.0011 nats |
| AMD GPU | MI350X 288 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| AMD GPU | MI350X 288 GB | Nemotron-3-Super-120B | Pass | Next-token KL divergence vs vLLM: 0.0015 nats |
| AMD GPU | MI300X 192 GB | Kimi-K3-2.8T | Pass | Next-token KL divergence vs vLLM: 0.0003 nats |
| AMD GPU | MI300X 192 GB | DeepSeek-V4.1-Flash-552B | Pass | Next-token KL divergence vs vLLM: 0.0180 nats |
| AMD GPU | MI300X 192 GB | DeepSeek-V4-Flash-284B | Fail | Next-token KL divergence vs vLLM: 0.3988 nats |
| AMD GPU | MI300X 192 GB | GLM-5.3-Flash-320B | Pass | Next-token KL divergence vs vLLM: 0.0021 nats |
| AMD GPU | MI300X 192 GB | Hy4-770B | Pass | Next-token KL divergence vs vLLM: 0.0011 nats |
| AMD GPU | MI300X 192 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| AMD GPU | MI300X 192 GB | Nemotron-3-Super-120B | Pass | Next-token KL divergence vs vLLM: 0.0044 nats |
| Google TPU | TPU v6e 32 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| Google TPU | TPU v6e 32 GB | Qwen3.8-27B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| Google TPU | TPU v6e 32 GB | Qwen3.6-35B | Pass | Next-token KL divergence vs vLLM: 0.0007 nats |
| Google TPU | TPU v6e 32 GB | Qwen3.6-27B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| Google TPU | TPU v6e 32 GB | Kimi-K3-2.8T | Pass | Next-token KL divergence vs vLLM: 0.0002 nats |
| Google TPU | TPU v6e 32 GB | DeepSeek-V4.1-Flash-552B | Pass | Next-token KL divergence vs vLLM: 0.0105 nats |
| Google TPU | TPU v6e 32 GB | DeepSeek-V4-Flash-284B | Pass | Next-token KL divergence vs vLLM: 0.0056 nats |
| Google TPU | TPU v6e 32 GB | Muse-Spark-Open-816B | Pass | Next-token KL divergence vs vLLM: 0.0002 nats |
| Google TPU | TPU v6e 32 GB | Muse-Glimmer-30B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| Google TPU | TPU v6e 32 GB | Gemma-4-31B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| Google TPU | TPU v6e 32 GB | Gemma-4-26B | Pass | Next-token KL divergence vs vLLM: 0.0031 nats |
| Google TPU | TPU v6e 32 GB | GPT-OSS-120B | Pass | Next-token KL divergence vs vLLM: 0.0386 nats |
| Google TPU | TPU v6e 32 GB | GLM-5.3-744B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| Google TPU | TPU v6e 32 GB | GLM-5.3-Flash-320B | Pass | Next-token KL divergence vs vLLM: 0.0060 nats |
| Google TPU | TPU v6e 32 GB | MiniMax-M3.1-Flash-428B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| Google TPU | TPU v6e 32 GB | Hy4-770B | Pass | Next-token KL divergence vs vLLM: 0.0009 nats |
| Google TPU | TPU v6e 32 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| Google TPU | TPU v6e 32 GB | Nemotron-3-Super-120B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| Google TPU | TPU v5e 16 GB | Qwen3.8-Flash-Next-125B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| Google TPU | TPU v5e 16 GB | Qwen3.8-27B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| Google TPU | TPU v5e 16 GB | Qwen3.6-35B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| Google TPU | TPU v5e 16 GB | Qwen3.6-27B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| Google TPU | TPU v5e 16 GB | Kimi-K3-2.8T | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| Google TPU | TPU v5e 16 GB | DeepSeek-V4.1-Flash-552B | Pass | Next-token KL divergence vs vLLM: 0.0142 nats |
| Google TPU | TPU v5e 16 GB | DeepSeek-V4-Flash-284B | Pass | Next-token KL divergence vs vLLM: 0.0055 nats |
| Google TPU | TPU v5e 16 GB | Muse-Spark-Open-816B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| Google TPU | TPU v5e 16 GB | Muse-Glimmer-30B | Pass | Next-token KL divergence vs vLLM: 0.0001 nats |
| Google TPU | TPU v5e 16 GB | Gemma-4-31B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| Google TPU | TPU v5e 16 GB | Gemma-4-26B | Pass | Next-token KL divergence vs vLLM: 0.0030 nats |
| Google TPU | TPU v5e 16 GB | Gemma-4-12B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| Google TPU | TPU v5e 16 GB | GPT-OSS-120B | Pass | Next-token KL divergence vs vLLM: 0.0139 nats |
| Google TPU | TPU v5e 16 GB | GPT-OSS-20B | Pass | Next-token KL divergence vs vLLM: 0.0020 nats |
| Google TPU | TPU v5e 16 GB | GLM-5.3-744B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| Google TPU | TPU v5e 16 GB | GLM-5.3-Flash-320B | Pass | Next-token KL divergence vs vLLM: 0.0059 nats |
| Google TPU | TPU v5e 16 GB | MiniMax-M3.1-Flash-428B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| Google TPU | TPU v5e 16 GB | Hy4-770B | Pass | Next-token KL divergence vs vLLM: 0.0009 nats |
| Google TPU | TPU v5e 16 GB | Nemotron-3-Ultra-550B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| Google TPU | TPU v5e 16 GB | Nemotron-3-Super-120B | Pass | Next-token KL divergence vs vLLM: 0.0000 nats |
| Browser | RTX 5090 Chrome 32 GB | Qwen3.8-27B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Browser | RTX 5090 Chrome 32 GB | Qwen3.6-27B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Browser | RTX 5090 Chrome 32 GB | Muse-Glimmer-30B | Pass | Next-token KL divergence vs MLX: 0.0001 nats |
| Browser | RTX 5090 Chrome 32 GB | Gemma-4-31B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Browser | RTX 5090 Chrome 32 GB | Gemma-4-26B | Pass | Next-token KL divergence vs MLX: 0.1224 nats |
| Browser | RTX 5090 Chrome 32 GB | GPT-OSS-120B | Pass | Next-token KL divergence vs MLX: 0.0016 nats |
| Browser | RTX 5090 Chrome 32 GB | GPT-OSS-20B | Pass | Next-token KL divergence vs MLX: 0.0001 nats |
| Browser | RTX 4090 Chrome 24 GB | Qwen3.8-27B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Browser | RTX 4090 Chrome 24 GB | Qwen3.6-27B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Browser | RTX 4090 Chrome 24 GB | Muse-Glimmer-30B | Pass | Next-token KL divergence vs MLX: 0.0001 nats |
| Browser | RTX 4090 Chrome 24 GB | Gemma-4-31B | Pass | Next-token KL divergence vs MLX: 0.0000 nats |
| Browser | RTX 4090 Chrome 24 GB | Gemma-4-26B | Pass | Next-token KL divergence vs MLX: 0.1371 nats |
| Browser | RTX 4090 Chrome 24 GB | GPT-OSS-120B | Pass | Next-token KL divergence vs MLX: 0.0014 nats |
| Browser | RTX 4090 Chrome 24 GB | GPT-OSS-20B | Pass | Next-token KL divergence vs MLX: 0.0002 nats |