Skip to main content

User and inferlet (session)

Every running inferlet has a session: a bidirectional channel to the client that launched it. The inferlet sends progress events, partial outputs, and files; the client sends messages, files, and signals. This page covers the session API. Read this after I/O overview.

Send to the client​

session.send ships text or structured data to the client. session.send_file ships binary blobs.

use inferlet::pie::core::session;

session::send("hello");                     // text
session::send_file(&pdf_bytes);             // binary

The client receives each send as a Stdout event and each send_file as a File event. See the client SDK reference for the receive side.

Receive from the client​

session.receive waits for the next message; session.receive_file waits for the next file. Both are async.

use inferlet::pie::core::session;
use inferlet::FutureStringExt;

let next = session::receive();
let msg: Option<String> = next.wait_async().await;   // None if the client disconnected

The receive futures resolve on the next inbound payload. If the client never sends, they wait forever; pair them with select! / asyncio.wait_for / Promise.race if you need a timeout.

Stream tokens as they decode​

The most common pattern: forward each chat-decoder delta to the session.

use inferlet::{chat, sample::Sampler};
use inferlet::pie::core::session;

let mut g = ctx.generate(Sampler::TopP { temperature: 0.6, p: 0.95 })
    .max_tokens(512);
let mut dec = chat::Decoder::new(&model);

while let Some(step) = g.next()? {
    let out = step.execute().await?;
    if let chat::Event::Delta(s) = dec.feed(&out.tokens)? {
        session::send(&s);
    }
}

The client UI updates token by token. The inferlet's final return value still arrives as the Return event after the loop ends.

Structured events​

Send a JSON object instead of a plain string when you want the client to dispatch on a tag:

use inferlet::pie::core::session;

let payload = serde_json::json!({
    "channel": "progress",
    "step": 3,
    "of": 10,
}).to_string();
session::send(&payload);

The client receives the JSON-encoded string and parses on its end. Common channels: progress, think, answer, tool-result. The convention is yours.

Receive files​

For inferlets that operate on user-uploaded data, the client uploads via the client SDK's transfer_file and the inferlet reads with session.receive_file.

use inferlet::pie::core::session;
use inferlet::FutureBlobExt;

let blob: Vec<u8> = session::receive_file()
    .wait_async()
    .await
    .ok_or("client disconnected")?;
process_pdf(&blob)?;

The client side uses process.transfer_file(blob) (Python) or its equivalent. See the client SDK reference.

Signals from the client​

A signal is a fire-and-forget string the client sends with process.signal(...). Inside the inferlet, signals arrive on the same channel as text messages, through session.receive. Treat them as out-of-band hints by tagging them on the client side:

// Client side: process.signal("stop").
// Inferlet side: pick the message up via session::receive.
use inferlet::pie::core::session;
use inferlet::FutureStringExt;

let next = session::receive();
if next.wait_async().await.as_deref() == Some("stop") {
    return Ok("stopped".into());
}

Signals are not a separate channel from the inferlet's perspective. If you need them distinguishable, encode the discriminator yourself (e.g. "signal:stop" or a JSON payload).

Stderr and the engine​

stderr writes from inside the inferlet (eprintln!, print(file=sys.stderr), console.error) reach the client as Stderr events. They are separate from session.send, so a UI can route them differently (typically to a log panel).

The pie run CLI mirrors Stderr events to the launcher's terminal. A long-running pie serve writes them to its own logs.

Next​