> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gatelit.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Streaming

> Stream token-by-token responses with async iteration. Cancel mid-stream, collect usage, and handle errors.

`stream()` returns an async generator that yields chunks as the model generates them. Each chunk carries an incremental text delta — ideal for chat UIs and live-completion experiences.

## Basic streaming

```ts theme={null}
const client = new GatelitClient({
  gatewayUrl: "https://gateway.yourapp.com",
  getToken: async () => ({ token: "sk_live_...", scheme: "GatelitKey" }),
})

for await (const chunk of client.stream({
  model: "anthropic/claude-3-5-haiku-20241022",
  messages: [{ role: "user", content: "Write a haiku about clouds." }],
})) {
  process.stdout.write(chunk.delta)
}
// → White mountains of air
// → Drifting across the blue sky
// → Rain waits in their shade
```

Each chunk has three fields:

| Field          | Type             | Description                                         |
| -------------- | ---------------- | --------------------------------------------------- |
| `delta`        | `string`         | Incremental text (empty on the final usage chunk)   |
| `finishReason` | `string \| null` | Set on the final chunk — `"stop"`, `"length"`, etc. |
| `usage`        | `Usage \| null`  | Token counts — only present on the final chunk      |

## Collecting the full response

Accumulate deltas across the stream to build the complete text:

```ts theme={null}
let fullText = ""
let usage = null

for await (const chunk of client.stream({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Explain TCP in one paragraph." }],
})) {
  fullText += chunk.delta
  if (chunk.usage) usage = chunk.usage
}

console.log(fullText)
console.log(`Tokens: ${usage?.totalTokens}`)
```

## Cancelling mid-stream

Pass an `AbortSignal` to stop generation early:

```ts theme={null}
const controller = new AbortController()

for await (const chunk of client.stream({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "List all prime numbers." }],
  signal: controller.signal,
})) {
  process.stdout.write(chunk.delta)
  if (fullText.length > 500) controller.abort()
}
```

## Updating a UI

The streaming API is designed for reactive UIs. Here's a React hook sketch:

```ts theme={null}
function useStream(model: string) {
  const [text, setText] = useState("")
  const [streaming, setStreaming] = useState(false)
  const controller = useRef<AbortController>()

  async function send(message: string) {
    setStreaming(true)
    setText("")
    controller.current = new AbortController()

    try {
      for await (const chunk of client.stream({
        model,
        messages: [{ role: "user", content: message }],
        signal: controller.current.signal,
      })) {
        setText(prev => prev + chunk.delta)
      }
    } finally {
      setStreaming(false)
    }
  }

  function stop() { controller.current?.abort() }

  return { text, streaming, send, stop }
}
```

Or use `useGatelit()` from `@gatelit/sdk-vue` for Vue — it handles all of this for you.

## Streaming a saved prompt

Streaming works with prompts too. Pass the prompt's slug as `promptId` for log correlation:

```ts theme={null}
for await (const chunk of client.stream({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Hi" }],
  promptId: "my-assistant",
})) {
  process.stdout.write(chunk.delta)
}
```

This tags the log entry with the prompt ID — the prompt's content is not used. To run a prompt's actual template with variable interpolation, use `/v1/prompts/:slug/run` via curl or the SDK's `chat()` with the `promptId` field pointing at the slug.