Ship it

An inferlet is a single .py or .js file, or a Rust crate compiled to WebAssembly. You can run it directly with pie run, or install it on a server and call it by name.

Python and JavaScriptCopy Anchor Link

A script inferlet is a single file (main.py or index.js). Pie runs it inside the WebAssembly language runtime installed in $PIE_HOME. To list installed language runtimes or install one manually:

pie language list
pie language install pie-language-python.tar.gz

RustCopy Anchor Link

A Rust inferlet is a cdylib crate compiled for wasm32-wasip3:

use inferlet::eta::prelude::*;

#[inferlet::main]
async fn main(input: String) -> Result<String, String> {
    // ...
    Ok(output)
}

cargo build --release --target wasm32-wasip3
pie run target/wasm32-wasip3/release/my_inferlet.wasm -- --prompt "Hi"

Rust inferlets produce small .wasm binaries with fast startup and expose the full SDK, including session::send_to and inferlet::speculative.

Install on a serverCopy Anchor Link

pie inferlet install main.py
pie inferlet list
pie inferlet remove my-inferlet

An installed inferlet is named after its file (main.py and index.js use their parent directory's name instead). Pass --version to set a version tag, and --force to overwrite an existing inferlet of the same name and version.

Call an inferletCopy Anchor Link

Arguments after -- become the JSON input object passed to main: --max-tokens 16 --verbose arrives as {"max_tokens": 16, "verbose": true}.

OutputsCopy Anchor Link

main returns a JSON object as the final result. Text sent with session.send() streams to the caller as messages during execution, and binary files sent with session.send_file() are written to the directory specified by pie run -o DIR.