Chat

chat() sends a request and waits for the full response. Use it when you need the complete output before continuing — data extraction, classification, or when streaming isn’t practical.

Basic usage

import { GatelitClient } from "@gatelit/sdk"

const client = new GatelitClient({
  gatewayUrl: "https://gateway.gatelit.dev",
  getToken: async () => ({ token: "glk_...", scheme: "GatelitKey" }),
})

const response = await client.chat({
  model: "openai/gpt-4o",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "What is the capital of Bhutan?" },
  ],
})

console.log(response.content)
// → "The capital of Bhutan is Thimphu."

console.log(response.usage)
// → { inputTokens: 28, outputTokens: 9, totalTokens: 37 }

The response

chat() returns a ChatResponse with everything you need:

Field	Type	Description
`content`	`string`	The assistant’s response text
`model`	`string`	Resolved model in `provider/model` format (the model that actually responded)
`usage`	`Usage \| null`	Token counts (`inputTokens`, `outputTokens`, `totalTokens`)
`finishReason`	`string \| null`	Why the model stopped — `"stop"`, `"length"`, `"tool_calls"`, or `"content_filter"`
`meta`	`GatelitMeta`	Request ID (`requestId`) and model — use for log lookup in the dashboard

Controlling generation

const response = await client.chat({
  model: "anthropic/claude-3-5-sonnet-20241022",
  messages: [{ role: "user", content: "Write a short story about a robot." }],
  maxTokens: 500,       // Cap response length
  temperature: 0.7,     // Control creativity (0 = deterministic, 2 = wild)
})

Parameters you don’t pass stay at provider defaults. The gateway handles provider-specific quirks automatically — temperature is stripped for reasoning models (o3, o3-mini, o4-mini), max_tokens is renamed to max_completion_tokens where needed.

Aborting a request

Pass an AbortSignal to cancel an in-flight request:

const controller = new AbortController()

setTimeout(() => controller.abort(), 5000) // Cancel after 5 seconds

try {
  const response = await client.chat({
    model: "openai/gpt-4o",
    messages: [{ role: "user", content: "Write a very long essay..." }],
    signal: controller.signal,
  })
} catch (err) {
  if (err.name === "AbortError") {
    console.log("Request was cancelled")
  }
}

Linking requests to the dashboard

Pass promptId to associate the request with a saved prompt for log correlation:

const response = await client.chat({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Hi" }],
  promptId: "my-translator",
})

The prompt’s content is not used — this just tags the log entry. Run a saved prompt’s actual template through /v1/prompts/:slug/run instead.

Fallback chains

Pass fallbackChain to survive provider outages, rate limits, and refusals — if the primary model fails, the gateway tries each fallback in order:

const response = await client.chat({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Summarize this ticket" }],
  fallbackChain: [
    { model: "anthropic/claude-3-5-haiku-20241022", triggers: ["rate_limit", "timeout"] },
    { model: "google/gemini-2.5-flash" }, // no triggers = fires on any error
  ],
})

if (response.meta.usedFallback) {
  console.log(`Handled by ${response.meta.fallbackModel} (${response.meta.fallbackReason})`)
}

Trigger values: rate_limit, provider_error, timeout, refusal_detected. Without triggers, the entry fires on any failure. Every model in the chain must support the prompt’s output format. When structured output is set, models that can’t produce the format — including models unknown to the catalog — are skipped at request time. The dashboard editor flags them while you build the chain.

Tracking end users

Pass endUserId to tag requests with a user identifier. This shows up in the logs so you can filter by user:

const response = await client.chat({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Summarize this ticket" }],
  endUserId: "customer_42",
})

endUserId travels in the x-gatelit-end-user-id header and is never sent to the provider. If you also want the provider to receive an end-user ID (the OpenAI-standard user body field), pass user:

const response = await client.chat({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Summarize this ticket" }],
  user: "customer_42", // forwarded to the provider AND used as the logged end-user ID
})

Attaching metadata

Pass metadata to attach arbitrary tags to a request — environment, feature, session, tenant. The gateway stores them on the log entry, so you can filter and debug by them in the dashboard:

const response = await client.chat({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Summarize this ticket" }],
  metadata: {
    environment: "production",
    feature: "doc-summariser",
    session_id: "sess_123",
    retry_count: 2,
  },
})

Metadata values must be strings, numbers, or booleans, and are stripped before the request reaches the provider.

Error handling

The SDK throws a GatelitError with three fields:

import { GatelitError } from "@gatelit/sdk"

try {
  await client.chat({ model: "openai/gpt-4o", messages: [...] })
} catch (err) {
  if (err instanceof GatelitError) {
    console.log(err.code)    // "unauthorized" | "gateway_error" | "provider_error"
    console.log(err.message) // Human-readable description
    console.log(err.status)  // HTTP status code
  }
}

TypeScript

TypeScript Reference

Vue

Basic usage

The response

Controlling generation

Aborting a request

Linking requests to the dashboard

Fallback chains

Tracking end users

Attaching metadata

Error handling

​Basic usage

​The response

​Controlling generation

​Aborting a request

​Linking requests to the dashboard

​Fallback chains

​Tracking end users

​Attaching metadata

​Error handling

Basic usage

The response

Controlling generation

Aborting a request

Linking requests to the dashboard

Fallback chains

Tracking end users

Attaching metadata

Error handling