Serve an API

pie serve speaks the OpenAI, Anthropic, and Gemini HTTP APIs on a single port. Existing clients work as soon as you change their base URL.

Start the serverCopy Anchor Link

pie serve

By default, the server listens on 127.0.0.1:8080, which accepts connections only from the local machine. To accept remote connections, set listen under [server] in the config file.

The model name comes from name under [model] in config.toml, which defaults to "default". GET /v1/models lists the active model.

OpenAICopy Anchor Link

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="unused")
stream = client.chat.completions.create(
    model="default",
    messages=[{"role": "user", "content": "Write a haiku about pie."}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

Chat Completions supports:

• tools and tool_choice
• response_format: JSON object mode, JSON Schema, and the structural_tag format used by vLLM and SGLang
• Streaming (stream=True), n, logprobs, and seed
• Sampling parameters: temperature, top_p, top_k, min_p, and presence, frequency, and repetition penalties
• reasoning_effort, with thinking tokens returned in reasoning_content

The HTTP compatibility layer does not store conversation state between requests, so /v1/responses rejects previous_response_id. Send the full conversation history each time; prefix caching reuses the KV cache from earlier turns without recomputing it.

AnthropicCopy Anchor Link

POST /v1/messages accepts text, base64 images, tools and tool_choice, thinking, stop sequences, sampling parameters, and streaming.

import anthropic

client = anthropic.Anthropic(base_url="http://127.0.0.1:8080", api_key="unused")
msg = client.messages.create(
    model="default",
    max_tokens=256,
    messages=[{"role": "user", "content": "Hello"}],
)
print(msg.content[0].text)

GeminiCopy Anchor Link

POST /v1beta/models/{model}:generateContent and :streamGenerateContent accept text, function calling, systemInstruction, responseSchema, thinkingConfig, and standard generation parameters.

curl "http://127.0.0.1:8080/v1beta/models/default:generateContent" \
  -H "Content-Type: application/json" \
  -d '{"contents": [{"parts": [{"text": "Hello"}]}]}'

Unsupported parametersCopy Anchor Link

When a request uses an unsupported feature, Pie returns HTTP 400 rather than ignoring the field. Unsupported features include provider-hosted tools (such as web search), remote image URLs (pass base64 data instead), and requesting multiple candidates on Anthropic or Gemini.

Pie's native endpointsCopy Anchor Link

The same port also serves Pie's native endpoints for running inferlets:

• POST /v1/generate runs an inferlet for one request over HTTP or server-sent events (SSE).
• GET /v1/ws opens a WebSocket that keeps an inferlet process alive across turns.

See Sessions and the HTTP & WebSocket reference.