Hardware
Run Pie on CUDA, ROCm, Metal, Vulkan, WebGPU, and Google TPUs.
Pie runs the same models and inferlets on datacenter GPUs, Google TPUs, laptops, phones, and browser tabs. Each backend has its own binary build, and the installer selects the matching build for your machine.
Backends
| Backend | Devices | Platforms | Multi-GPU |
|---|---|---|---|
| CUDA | NVIDIA GPUs | Linux, Windows | Tensor and expert parallelism |
| ROCm | AMD GPUs | Linux | Tensor and expert parallelism |
| Metal | Apple silicon | macOS, iOS | Single device |
| Vulkan | Cross-vendor GPUs, including Android phones | Linux, Windows, Android | Single device |
| WebGPU | Web browsers, and native via wgpu | Chrome, Linux, macOS, Windows | Single device |
| XLA | Google TPUs | Linux | Single device |
All prebuilt binaries are listed on the install page.
Choosing a device
pie config init writes an [engine] section for your installed backend. On a single-GPU machine, the defaults work without changes:
[engine]
type = "cuda"
device = ["cuda:0"]
activation_dtype = "bfloat16"To select a different GPU, change device, for example ["cuda:1"] or ["vulkan:1"]. Device numbers follow the driver's enumeration order.
Running on multiple GPUs
On CUDA and ROCm, list the target GPUs and set how the model is sharded across them:
[engine]
type = "cuda"
device = ["cuda:0", "cuda:1", "cuda:2", "cuda:3"]
tensor_parallel_size = 4tensor_parallel_sizeshards each layer's weight matrices across GPUs.expert_parallel_sizeshards Mixture-of-Experts (MoE) experts across GPUs instead.
Import a deployment that matches your GPU count, such as one ending in -tp4 (see Models). To scale across multiple machines, see Scale out.
Check your setup
pie doctorpie doctor verifies your config file, imported models, engine libraries, and GPU drivers, and prints what to fix if a check fails. After pie doctor passes, run pie tune once to benchmark your device. See Configure & tune.
See how Pie shares one runtime across thin per-device engines and generated kernels.