Cookbook

More than fifty complete inferlets you can run, read, and copy.

Every recipe below is a complete inferlet in the Pie repository. Clone the repository to run any recipe by name, or copy its source as a starting point for your own inferlet:

git clone https://github.com/pie-project/pie && cd pie/examples
pie run top-a-sampling-py -- --prompt "Once upon a time" --a 0.2

The colored dot marks which layer of the stack the recipe uses: ● uses the Program layer, and ● uses Invent primitives such as multi-lane passes, custom attention masks, or device loops. The language tags link to each implementation on GitHub.

Basics

quickstart

Greedy completion, one pass per token, driven from the host.

PythonJS
text-completion

Greedy completion with the decode loop carried on the device.

RustPythonJS
chat-completion

A chat reply with temperature and top-p, streamed.

Rust
naive-baseline

Plain sampling, the performance control for the other recipes.

RustPythonJS

Sampling

top-a-sampling

Keep tokens above a · p_max², a floor that tracks confidence.

RustPythonJS
xtc-sampling

Exclude top choices, for more varied text.

Rust
tail-free-sampling

Cut the tail where the sorted probabilities flatten.

Rust
locally-typical-sampling

Keep tokens close to the expected surprise.

Rust
mirostat-v2-sampling

Steer the surprise of the output toward a target.

Rust
eta-epsilon-sampling

Epsilon and eta truncation.

Rust
entropy-adaptive-temperature

Raise or lower the temperature with the entropy.

Rust
repetition-penalty

Frequency, presence and repetition penalties.

Rust
dry-repetition-penalty

Penalize tokens that would extend a repeated sequence.

Rust

Guidance

classifier-free-guidance

Two lanes per pass: with and without the prompt.

Rust
contrastive-decoding

Contrastive decoding with a single model.

Rust
context-aware-decoding

Trust the given context over the model’s prior.

Rust

Search and branching

beam-search

Beam search on the device, with ancestry masks.

RustPythonJS
consensus-decoding

Generate N answers from one prefix and pick the most central.

Rust
prefix-tree-kv-cache

A tree of prompts sharing their common prefixes.

RustPythonJS

Speculative decoding

mtp-speculative-decoding

Draft from the model’s own MTP head; verify and commit on the device.

Rust
cacheback-speculative-decoding

Prompt-lookup drafts from earlier in the context.

Rust
constrained-speculative-decoding

Speculation under a JSON Schema.

Rust
dflash-speculative-bench

A block drafter’s draft-then-verify loop.

Rust

KV cache

attention-sink

Attention sinks plus a sliding window.

Rust
snapkv-eviction

Use the prompt’s attention to shrink the cache.

Rust
tova-attention

Drop the entry attended to least.

Rust
trackb-h2o

Keep the heavy hitters.

Rust
trackb-snapkv

SnapKV from an observation window.

Rust

Constrained output

json-schema-constrained-decoding

JSON that always matches a schema.

RustPythonJS
asap-grammar-aligned-decoding

Grammar-constrained sampling without the distribution bias.

Rust
token-healing

Repair prompts that end mid-token.

Rust

Watermarking

greenlist-watermarking

Bias a keyed greenlist of tokens.

Rust
gumbel-watermark

A distortion-free watermark (Aaronson).

Rust
synthid-tournament-sampling

SynthID-Text tournament sampling.

Rust

Images, video and diffusion

text-to-media

Text to image or video with any diffusion model.

Rust
image-captioning

Describe an image.

Rust
diffusion-baseline

A text diffusion model’s reference sampler.

Rust