# Model slices

> Single-device kernel benchmarking for frontier models that exceed accelerator memory.

Source: https://pie-project.org/benchmarks/methodology/slices  

Frontier MoE and dense checkpoints often exceed the memory of a single GPU or Mac, yet single-accelerator kernel efficiency — attention, quantized GEMMs, MoE dispatch, and graph launch overhead — can still be isolated without multi-node confounding. A **model slice** (marked `▫` in the grids) trims total parameter storage to fit one device while preserving the exact per-layer compute geometry of the full architecture.

## How slices are constructed

Slicing never alters hidden dimensions, attention head counts, head dimensions, MoE intermediate widths, active experts per token (`top-k`), quantization group sizes, or the vocabulary projection head:

1. **Expert pruning first:** For Mixture-of-Experts models, total stored routed experts are reduced while keeping `top-k` active experts per token unchanged. Every token dispatches the same arithmetic volume and active weight footprint per layer as the full model (aside from higher L2/DRAM cache locality on expert weights, which applies equally to every engine).
2. **Whole-block layer reduction second:** Only if expert pruning alone still exceeds target VRAM are repeating transformer blocks removed, always in complete architectural periods (for example, preserving hybrid linear/full-attention ratios).

Every engine in a cell loads the exact same slice checkpoint, processes identical token IDs for all 256 decode steps, and undergoes [next-token KL verification](https://pie-project.org/benchmarks/methodology/quality) against a reference engine.

## Scope and limitations

Slices are strictly a **controlled relative benchmark** on a single device:

* **Omitted from absolute `tok/s` views:** Because fewer layers execute per token (or fewer experts contend for capacity), raw `tok/s` on a slice is faster than the full model and is never reported in the **Throughput (tok/s)** view.
* **Not a multi-GPU proxy:** Multi-GPU and multi-node throughput depends on tensor-parallel all-reduce latency and expert-parallel all-to-all communication, which single-device slices do not exercise.
* **Excluded from end-to-end tasks:** Downstream accuracy benchmarks (MATH-500) and speculative draft acceptance rates require coherent semantics and run exclusively on unmodified checkpoints.

## Representative slice configurations

| Model              | Layers retained | Routed experts retained | Target memory tier   |
| ------------------ | --------------- | ----------------------- | -------------------- |
| GLM-5.3            | 78 / 78         | 12 / 256                | 80 GB GPUs           |
| MiMo-V2.6-Flash    | 48 / 48         | 128 / 256               | 141 GB GPUs          |
| DeepSeek-V4-Flash  | 43 / 43         | 16 / 256                | Single-GPU tiers     |
| Kimi-K3            | 8 / 93          | 32 / 896                | Single-GPU tiers     |
| Qwen3.8-Flash-Next | 4               | 16 / 512                | Consumer / Mac tiers |
| Gemma-4-31B        | 12 / 60         | Dense (no MoE)          | 24 GB GPUs           |