Chat and generation
The starting point: build a chat prompt, prefill it, and decode with top-p and temperature sampling.
| Inferlet | What it shows |
|---|---|
chat-completion | Prompt prefill and device-carried top-p/temperature decoding with chat::Decoder. |
Related guide
- Generation overview: the three-layer model that these inferlets drive.
- Generator: the multi-step loop they all use.
- Chat parser: the streaming text parser the chat-style examples feed.