The forward pass
Lanes, state geometry, attention masks, readouts, and adapters.
Every inferlet is built on ForwardPass. How Pie thinks showed its basic form: one sequence of tokens, one working set, and one epilogue. This page covers the remaining controls on a forward pass.
Lanes
As the figure above shows, a single ForwardPass can carry multiple sequences at once, called lanes. Each lane has its own tokens and its own working set, and all lanes run in one GPU launch. Pass the concatenated tokens with a row offset array (indptr), and pass the working sets in the same order:
fwd = ForwardPass()
fwd.embed(cond + uncond, indptr=[0, len(cond), len(cond) + len(uncond)])
fwd.state([ws_cond, ws_uncond], StateGeometry.folding())Guidance methods use lanes to compare two contexts at every token: classifier-free guidance, contrastive decoding, and context-aware decoding. Inside the epilogue, intrinsics.output(lane, "logits") reads the logits for a specific lane.
State geometry
StateGeometry specifies where each input token ("row") writes its KV state in the working set, and how many slots are frozen after the pass finishes. When you use StateGeometry.folding(), rows are appended causally and frozen immediately.
For algorithms that manage KV slots directly, you can set four fields on StateGeometry:
| Field | Scope | Meaning |
|---|---|---|
positions | Per row | Position ID used for RoPE encoding. |
slots | Per row | Slot index in the working set where the row writes its KV state. |
kv_len | Per lane | Number of KV slots the lane reads during this pass. |
fold | Per lane | Number of newly written slots to freeze after the pass: none, all, or a count. |
In speculative decoding, for example, the pass writes draft tokens without freezing them (fold=0). After verification, the epilogue writes the accepted count into a device channel, so the GPU freezes only the accepted prefix without waiting on the host.
Attention masks
fwd.mask(rows) specifies, for each input row, which KV slots it attends to. A row can skip slots it would normally read under a causal mask. That is all you need to implement sliding-window attention, attention sinks, and attention-based KV eviction such as TOVA. On hybrid models with recurrent layers, fwd.mask applies to the attention layers.
Readouts
By default, an epilogue receives logits only for the last row of each lane. fwd.readout(indices) selects other rows instead. For example, speculative verification reads out every position of a draft window, and prompt scoring reads out every prompt token.
Adapters
fwd.adapter(site, f) attaches an adapter at a projection site ("q", "k", "v", "o", "gate_up", or "down"). Your function f(x, y) takes the projection input x and base output y and returns the modified output expression. This single interface covers LoRA (y + b * (a * x)), IA3 (y scaled), and DoRA. Concurrent requests in the same batch can use different adapters.
Pipelines and frames
Passes on one Pipeline run in submission order. Because submit does not block, you can queue the next pass before the previous one finishes. Use a separate pipeline for each independent stream, such as each branch of a search tree.
pipe.submit_frame([...]) submits multiple passes to run in back-to-back scheduler waves. pipe.run_ahead(fwd, budget, on_token) keeps a repeating decode pass in flight on the GPU. See Device epilogues.
Compile custom sampling and verification code into GPU kernels.