Quickstart

You can start with Pie in three ways depending on what you want to build.

The first two paths require Pie installed and one model imported:

pie config init
pie model import Qwen/Qwen3.5-0.8B

Call it like OpenAICopy Anchor Link

Start the server. It listens on 127.0.0.1:8080 by default.

pie serve

Call it with any OpenAI client. Use "default" as the model name, which matches the [model] section created by pie config init:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="unused")
reply = client.chat.completions.create(
    model="default",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
)
print(reply.choices[0].message.content)

The same port also serves the Anthropic and Gemini APIs. See Serve an API.

Run an inferletCopy Anchor Link

An inferlet is a program that runs inside the engine and drives the model one forward pass at a time. Save this script as main.py:

from inferlet import chat, model
from inferlet.eta import ForwardPass, Pipeline, StateGeometry, WorkingSet, intrinsics, reduce_argmax


async def main(input: dict) -> dict:
    tokens = chat.prefix() + model.encode(input.get("prompt", "The capital of France is"))
    max_tokens = int(input.get("max_tokens", 8))
    ws = WorkingSet()
    pipe = Pipeline()

    done = 0
    for _ in range(max_tokens):
        fwd = ForwardPass()
        fwd.embed(tokens[done:])
        fwd.state([ws], StateGeometry.folding())
        out = fwd.epilogue(lambda: reduce_argmax(intrinsics.output(0, "logits")))
        pipe.submit(fwd)
        done = len(tokens)
        tokens.append(await out)
    pipe.close()

    return {"text": model.decode(tokens[-max_tokens:])}

Run it with pie run. Flags after -- become keys in the input dictionary, so --max-tokens 16 arrives as {"max_tokens": 16}:

pie run main.py -- --prompt "The capital of France is" --max-tokens 16

You control every step of that loop: what tokens go in, how the next token is chosen, and when to stop. How Pie thinks explains each primitive, and Your first inferlet extends this example with chat templates and streaming.

Try it in your browserCopy Anchor Link

Open the Playground. It downloads a small model once and runs Pie on your GPU through WebGPU. You can edit the example inferlets and re-run them in the tab.

Where nextCopy Anchor Link