# Invent

> Low-level primitives: forward passes, device epilogues, KV cache control, scheduling policies, and Poem models.

Source: https://pie-project.org/docs/invent  

You can implement new inference algorithms and model architectures in Pie without modifying the engine. Built-in features such as the OpenAI API, speculative decoding, and cache-aware routing are written using the same primitives described in this section.

> **Figure.** Inside Pie: inferlets submit passes to the runtime, whose scheduling policy decides what runs; the eta compiler turns epilogues into kernels; the Poem model defines the forward pass; kernels run on each backend. Inferlets, policies, epilogues and models are all replaceable programs. Dashed outlines are programs you can write: inferlets, epilogues, policies and models.

As the figure above shows, four programmable pieces meet here:

* **Inferlets** control the generation loop: what each `ForwardPass` reads and writes, and how its outputs are used.
* **Epilogues** run on the GPU right after each forward pass, compiled from Python, JavaScript, or Rust.
* **Policies** configure which process runs next, which KV pages stay cached, and how requests route across a cluster.
* **Models** are Poem files that define the forward pass itself.

- [The forward pass](https://pie-project.org/docs/invent/forward-pass): Lanes, state geometry, attention masks, readouts, and adapters.

- [Device epilogues](https://pie-project.org/docs/invent/epilogues): Samplers compiled to GPU kernels, and decode loops that stay on the device.

- [Speculative decoding](https://pie-project.org/docs/invent/speculative-decoding): Draft, verify, and commit tokens as an inferlet.

- [KV cache](https://pie-project.org/docs/invent/kv-cache): Shared pages, eviction algorithms, and memory tiers.

- [Scheduling policies](https://pie-project.org/docs/invent/scheduling): Configure how work is ordered, cached, and routed.

- [Write a model in Poem](https://pie-project.org/docs/invent/poem): Define or edit a model architecture in a short file.

- [Backends & kernels](https://pie-project.org/docs/invent/backends): How one codebase targets CUDA, ROCm, Metal, Vulkan, WebGPU, and TPUs.

The [Cookbook](https://pie-project.org/docs/cookbook) contains more than fifty complete inferlets covering samplers, guidance, tree search, speculative decoding, KV eviction, and watermarks.
