Speculative decoding
Inferlets that implement and verify draft tokens before committing them.
Running on the dummy driver?
The dummy driver samples a fresh random token for every slot, so every draft is rejected and these inferlets run in 1-token-per-step fallback — they complete correctly but no speedup is observable.
| Inferlet | What it shows |
|---|---|
cacheback-speculative-decoding | Prompt-lookup n-gram drafting with batched target verification. |
mtp-speculative-decoding | Native MTP drafting with explicit accept, correction, and bonus-token handling. |
Related guide
- Speculative decoding: the
Speculatortrait, system speculation, and the verify path. - The forward pass: position IDs that drafts share with their target.