Chat

chat() sends a request and waits for the full response. Use it when you need the complete output before continuing — data extraction, classification, or when streaming isn’t practical.

Basic usage

import { GatelitClient } from "@gatelit/sdk"

const client = new GatelitClient({
  gatewayUrl: "https://gateway.gatelit.dev",
  getToken: async () => ({ token: "glk_...", scheme: "GatelitKey" }),
})

const response = await client.chat({
  model: "openai/gpt-4o",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "What is the capital of Bhutan?" },
  ],
})

console.log(response.content)
// → "The capital of Bhutan is Thimphu."

console.log(response.usage)
// → { inputTokens: 28, outputTokens: 9, totalTokens: 37 }

The response

chat() returns a ChatResponse with everything you need:

Field	Type	Description
`content`	`string`	The assistant’s response text
`model`	`string`	Resolved model in `provider/model` format (the model that actually responded)
`usage`	`Usage \| null`	Token counts (`inputTokens`, `outputTokens`, `totalTokens`)
`finishReason`	`string \| null`	Why the model stopped — normally `"stop"`, `"length"`, `"tool_calls"`, or `"content_filter"`; provider-specific reasons not in that set (e.g. Anthropic `"end_turn"`) are passed through lowercased
`meta`	`GatelitMeta`	Request ID (`requestId`) and model — use for log lookup in the dashboard

Controlling generation

const response = await client.chat({
  model: "anthropic/claude-3-5-sonnet-20241022",
  messages: [{ role: "user", content: "Write a short story about a robot." }],
  maxTokens: 500,       // Cap response length
  temperature: 0.7,     // Control creativity (0 = deterministic, 2 = wild)
})

Parameters you don’t pass stay at provider defaults. The gateway handles provider-specific quirks automatically — temperature is stripped for reasoning models (o3, o3-mini, o4-mini), max_tokens is renamed to max_completion_tokens where needed.

Aborting a request

Pass an AbortSignal to cancel an in-flight request:

const controller = new AbortController()

setTimeout(() => controller.abort(), 5000) // Cancel after 5 seconds

try {
  const response = await client.chat({
    model: "openai/gpt-4o",
    messages: [{ role: "user", content: "Write a very long essay..." }],
    signal: controller.signal,
  })
} catch (err) {
  if (err.name === "AbortError") {
    console.log("Request was cancelled")
  }
}

Linking requests to the dashboard

Pass promptId to associate the request with a saved prompt for log correlation:

const response = await client.chat({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Hi" }],
  promptId: "my-translator",
})

The prompt’s content is not used — this just tags the log entry. Run a saved prompt’s actual template through /v1/prompts/:slug/run instead.

Fallback chains

Pass fallbackChain to survive provider outages, rate limits, and refusals — if the primary model fails, the gateway tries each fallback in order:

const response = await client.chat({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Summarize this ticket" }],
  fallbackChain: [
    { model: "anthropic/claude-3-5-haiku-20241022", triggers: ["rate_limit", "timeout"] },
    { model: "google/gemini-2.5-flash" }, // no triggers = fires on any error
  ],
})

if (response.meta.usedFallback) {
  console.log(`Handled by ${response.meta.fallbackModel} (${response.meta.fallbackReason})`)
}

Trigger values: rate_limit, provider_error, timeout, refusal_detected. Without triggers, the entry fires on any failure. Every model in the chain must support the prompt’s output format. When structured output is set, models that can’t produce the format — including models unknown to the catalog — are skipped at request time. The dashboard editor flags them while you build the chain.

Tracking end users

Pass endUserId to tag requests with a user identifier. This shows up in the logs so you can filter by user:

const response = await client.chat({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Summarize this ticket" }],
  endUserId: "customer_42",
})

endUserId travels in the x-gatelit-end-user-id header and is never sent to the provider. If you also want the provider to receive an end-user ID (the OpenAI-standard user body field), pass user:

const response = await client.chat({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Summarize this ticket" }],
  user: "customer_42", // forwarded to the provider AND used as the logged end-user ID
})

Attaching metadata

Pass metadata to attach arbitrary tags to a request — environment, feature, session, tenant. The gateway stores them on the log entry, so you can filter and debug by them in the dashboard:

const response = await client.chat({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Summarize this ticket" }],
  metadata: {
    environment: "production",
    feature: "doc-summariser",
    session_id: "sess_123",
    retry_count: 2,
  },
})

Metadata values must be strings, numbers, or booleans, and are stripped before the request reaches the provider.

Response caching

Opt into exact-match caching for deterministic requests — non-streaming, temperature: 0, no tools. Identical requests are served from the gateway cache with no upstream call and no provider spend. Only successful responses are cached, so failure retries always hit the provider fresh.

const response = await client.chat({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Classify this email: order placed" }],
  temperature: 0,
  cache: true,              // default TTL (5 minutes)
  // cache: { ttl: 3600 }   // custom TTL, clamped to 60–3600s
  // cache: false           // bypass — e.g. an explicit retry for a fresh answer
})

if (response.meta.cacheHit) {
  console.log("served from cache")
}

Error handling

The SDK throws a GatelitError with three fields:

import { GatelitError } from "@gatelit/sdk"

try {
  await client.chat({ model: "openai/gpt-4o", messages: [...] })
} catch (err) {
  if (err instanceof GatelitError) {
    console.log(err.code)    // "unauthorized" | "gateway_error" | "provider_error"
    console.log(err.message) // Human-readable description
    console.log(err.status)  // HTTP status code
  }
}

TypeScript

TypeScript Reference

Vue

Basic usage

The response

Controlling generation

Aborting a request

Linking requests to the dashboard

Fallback chains

Tracking end users

Attaching metadata

Response caching

Error handling

​Basic usage

​The response

​Controlling generation

​Aborting a request

​Linking requests to the dashboard

​Fallback chains

​Tracking end users

​Attaching metadata

​Response caching

​Error handling

Basic usage

The response

Controlling generation

Aborting a request

Linking requests to the dashboard

Fallback chains

Tracking end users

Attaching metadata

Response caching

Error handling