Config
Every config section and its main keys.
pie config init writes a commented config.toml with the sections below. Every key has a default value; delete a line to restore its default. Sizes and durations include units ("4GiB", "50ms"). pie config list prints every key supported by your installed binary along with its current value and description.
[server]
| Key | Default | Meaning |
|---|---|---|
listen | "127.0.0.1:8080" | Bind address for HTTP and WebSocket connections. |
verbose | false | Log every request. |
worker_threads | CPU count (at most 64) | Number of runtime worker threads. |
max_upload | "256MiB" | Maximum upload size. |
[model]
| Key | Default | Meaning |
|---|---|---|
name | "default" | Model name exposed to clients. |
source | — | Hugging Face ID of the imported model. |
weight_dtype | From checkpoint | Weight dtype loaded from the .zt file. |
device_memory_share | 0.90 | Fraction of free device memory Pie reserves. |
device_kv_budget | Remaining share | Device memory allocated to KV cache pages. |
device_weight_budget | Uncapped | Optional cap on device memory used by weights. |
host_kv_budget | — | Host RAM budget for evicted KV pages. |
disk_kv_budget | — | Disk budget for evicted KV pages; "0B" disables disk offloading. |
disk_kv_dir | Under $PIE_HOME | Directory for the disk KV tier. |
disk_kv_persistent | false | Keep disk-tier KV pages across server restarts. |
[engine]
| Key | Meaning |
|---|---|
type | cuda, metal, vulkan, wgpu, xla, or cerebras. |
device | List of devices, such as ["cuda:0", "cuda:1"]. |
activation_dtype | Activation compute dtype, usually "bfloat16". |
tensor_parallel_size | Number of GPUs to shard each layer across (CUDA, ROCm). |
expert_parallel_size | Number of GPUs to shard MoE experts across (CUDA, ROCm). |
kv_page_size | Tokens per KV cache page; derived automatically when omitted. |
max_model_len | Maximum sequence length; defaults to the model's limit. |
max_state_slots | Maximum recurrent-state slots for hybrid models. |
max_forward_tokens, max_forward_requests | Per-batch token and request limits (Metal, Vulkan, WebGPU, XLA). |
Each backend also accepts backend-specific keys, such as pipeline_cache for Vulkan and WebGPU, backends and power_preference for WebGPU, or plugin for XLA. An unknown key produces an error naming the active backend.
[runtime]
| Key | Default | Meaning |
|---|---|---|
submit_deadline | "50ms" | Maximum time a batch waits to fill before launching. |
silence_timeout | "30s" | Inactivity timeout before a silent process is treated as stuck. |
max_concurrent_processes | From engine | Maximum number of inferlets running at once. |
worker_policy | "noop" | Active worker scheduling policy. |
[sandbox]
| Key | Default | Meaning |
|---|---|---|
allow_fs | false | Allow inferlets to access the scratch filesystem. |
allow_network | true | Allow inferlets to access the network. |
network_allowed_hosts | ["*"] | Hosts reachable by inferlets when allow_network = true. |
scratch_dir | — | Host directory mounted as scratch space for inferlets. |
max_memory | "4GiB" | Memory limit per inferlet instance. |
max_instances | 1000 | Maximum concurrent inferlet instances. |
[cluster]
| Key | Meaning |
|---|---|
cluster_policies | Active cluster policies. |
controller | Controller address for a distributed worker. |
role | Worker role: "decode" or "prefill". |
gateways | Gateway addresses this worker connects to. |
[offload]
| Key | Meaning |
|---|---|
listen | Address where this worker accepts prefill work from peers. |
peers | Peer workers to send prefill work to. |
pools | pie-pool nodes used to store and fetch KV pages. |
transfer | "auto", or "nixl" to read peer pages over RDMA. |
prefill_min_suffix_tokens | Minimum uncached suffix length required before offloading prefill to a peer. |
[telemetry]
| Key | Meaning |
|---|---|
endpoint | OTLP collector URL, such as "http://localhost:4317". |
service_name | Service name reported to the collector. |