Inferlet and inferlet (messaging)

Two inferlets running in the same engine talk through the messaging surface. There are two patterns: topic broadcast/subscribe (every subscriber sees every message) and queues (each message goes to one consumer). Read this after I/O overview.

The messaging bus is in-memory and engine-local. Messages do not cross engine instances; for cross-engine coordination, route through external infrastructure.

Broadcast and subscribe

Pub/sub on a named topic. Every subscriber sees every message.

Rust

use inferlet::{messaging, SubscriptionExt};

// Publisher
messaging::broadcast("events", "new-data");

// Subscriber
let sub = messaging::subscribe("events");
loop {
    if let Some(msg) = sub.get_async().await {
        eprintln!("got {msg}");
    }
}
sub.unsubscribe();

Python

from inferlet import messaging

# Publisher
messaging.broadcast("events", "new-data")

# Subscriber
sub = messaging.subscribe("events")
async for msg in sub:               # the Subscription is async-iterable
    print(f"got {msg}")
sub.unsubscribe()

# Or pull one at a time:
msg = await sub.next()              # returns str | None

JavaScript

import { messaging } from 'inferlet';

// Publisher
messaging.broadcast('events', 'new-data');

// Subscriber
const sub = messaging.subscribe('events');
for await (const msg of sub) {       // the Subscription is async-iterable
    console.error(`got ${msg}`);
}
sub.unsubscribe();

// Or pull one at a time:
const msg = await sub.next();        // returns string | undefined

Subscribers join with subscribe(topic). Publishers fire-and-forget with broadcast(topic, message). There is no backpressure: a slow subscriber drops messages. Topics are namespaced by string equality; pick names that won't collide.

The agent-swarm inferlet uses pub/sub to coordinate worker generations.

Queues

Each message goes to exactly one consumer. Use this for work distribution.

Rust

use inferlet::{messaging, FutureStringExt};

// Producer
messaging::push("jobs", &job_payload);

// Consumer
let fut = messaging::pull("jobs");
let msg: Option<String> = fut.wait_async().await;

Python

from inferlet import messaging

# Producer
messaging.push("jobs", job_payload)

# Consumer
msg = await messaging.pull("jobs")

JavaScript

import { messaging } from 'inferlet';

// Producer
messaging.push('jobs', jobPayload);

// Consumer
const msg = await messaging.pull('jobs');

Each pull consumes one message. Topic order is preserved (FIFO). Messages waiting in the queue when no consumer is pulling stay queued.

Patterns

• Sub-agent spawning. Parent inferlet pushes work items to a jobs queue; worker inferlets pull and process. The agent-swarm example shows the pattern.
• Coordination across forks. Two forks of a branching context exchange intermediate scores or signals. Useful for tree search where siblings prune each other.
• Cross-tenant signaling. Publish a "config-changed" event so cooperating inferlets reload state.
• Deferred work. A user-facing inferlet pushes a long task to a queue and returns immediately; a worker pulls and runs the task.

Lifecycle

Subscriptions and queues live as long as their topic has at least one user. When everyone unsubscribes from a topic and no messages are pending, the topic disappears. There is no admin to clean up dead subscriptions on inferlet exit, so make sure to call unsubscribe() (or let the SDK clean up on drop) before returning.

The bus is engine-local: shutdowns drop everything.

Next

• Filesystem: the durable counterpart to in-memory messaging.
• agent-swarm: a multi-agent inferlet coordinating workers via messaging.
• User and inferlet (session): the client-facing channel.