Chat Completions

curl https://gateway.gatelit.dev/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: GatelitKey glk_your_service_key" \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [{"role": "user", "content": "What is the capital of France?"}]
  }'

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1717776000,
  "model": "openai/gpt-4o",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The capital of France is Paris."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 8,
    "total_tokens": 23
  }
}

{
  "error": true,
  "code": "gateway_error",
  "message": "Missing or invalid 'model' field. Expected format: 'provider/model-name'"
}

{
  "error": true,
  "code": "unauthorized",
  "message": "Invalid or expired token"
}

{
  "error": true,
  "code": "provider_error",
  "message": "Upstream error 503 — Service Unavailable"
}

POST

chat

completions

curl https://gateway.gatelit.dev/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: GatelitKey glk_your_service_key" \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [{"role": "user", "content": "What is the capital of France?"}]
  }'

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1717776000,
  "model": "openai/gpt-4o",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The capital of France is Paris."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 8,
    "total_tokens": 23
  }
}

{
  "error": true,
  "code": "gateway_error",
  "message": "Missing or invalid 'model' field. Expected format: 'provider/model-name'"
}

{
  "error": true,
  "code": "unauthorized",
  "message": "Invalid or expired token"
}

{
  "error": true,
  "code": "provider_error",
  "message": "Upstream error 503 — Service Unavailable"
}

POST /v1/chat/completions Send a chat request to any supported provider through the gateway. The response format is identical to OpenAI’s chat completions API regardless of which provider handles the request.

Auth

All three schemes are accepted:

Authorization: GatelitKey glk_...
Authorization: GatelitSigned <signed-token>

Bearer (OIDC) is supported but in Alpha. Contact us if you need OIDC auth.

Request

Field	Type	Required	Description
`model`	`string`	Yes	Provider/model in `provider/model-id` format
`messages`	`array`	Yes	Array of `{role, content}` objects
`max_tokens`	`integer`	No	Max output tokens
`temperature`	`number`	No	0–2, stripped for reasoning models
`stream`	`boolean`	No	Enable SSE streaming
`user`	`string`	No	End-user ID — forwarded to the provider and used as the logged end-user ID when no token claim or `x-gatelit-end-user-id` header is present
`metadata`	`object`	No	Arbitrary string/number/boolean tags — stripped before forwarding, stored on the log entry, filterable in the dashboard

Content blocks

messages[].content may be a string or an array of content blocks:

{
  "role": "user",
  "content": [
    { "type": "text", "text": "Summarise this document" },
    {
      "type": "file",
      "file": { "mime_type": "application/pdf", "file_data": "data:application/pdf;base64,..." }
    }
  ]
}

Block	Shape
`text`	`{ "type": "text", "text": "..." }`
`image_url`	`{ "type": "image_url", "image_url": { "url": "https://… or data:image/…" } }`
`file`	`{ "type": "file", "file": { "mime_type": "...", "file_data" \| "url" \| "file_id": "..." } }`

Remote URLs are fetched server-side with SSRF checks and a shared deadline. Text files are inlined as text on every model; images and documents require a model with the matching capability. Anthropic accepts images and PDF documents only. See prompt file variables for the dashboard upload flow.

Gateway-specific fields

Field	Description
`output_schema`	Structured output config (`json_object` or `json_schema` mode)
`prompt_id`	Tag request with a saved prompt’s slug for log correlation
`fallback_chain`	Ordered fallback models with trigger conditions
`cache`	Opt-in response cache — `true`, `{ ttl: seconds }`, or `false` to bypass

Metadata

metadata lets you attach business context to a request — environment, feature, session, tenant, experiment — without polluting the provider call:

{
  "model": "openai/gpt-4o",
  "messages": [{"role": "user", "content": "Hello!"}],
  "metadata": {
    "environment": "production",
    "feature": "doc-summariser",
    "session_id": "sess_123"
  }
}

Rules:

Values must be string, number, or boolean — nested objects and arrays are rejected (400)
Keys are limited to 64 characters and values to 128 characters; at most 100 keys per request
The field is stripped before forwarding to the provider — it never reaches the upstream API
It appears on the log entry and can be filtered with key=value in the dashboard

The gateway also accepts user in the body — a standard OpenAI field. It is forwarded to OpenAI-compatible providers unchanged, and used as the logged end-user ID when no signed-token claim or x-gatelit-end-user-id header set one. (Non-OpenAI providers that lack the field — e.g. Anthropic, Google — ignore it; the gateway still logs it.)

Response caching

cache opts into an exact-match response cache. Only deterministic requests are eligible — non-streaming, explicit temperature: 0, no tools — and only successful responses are stored, so a failure retry always hits the upstream provider fresh.

curl https://gateway.gatelit.dev/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: GatelitKey glk_..." \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [{"role": "user", "content": "Classify this email: order placed"}],
    "temperature": 0,
    "cache": true
  }'

true — cache with the default TTL (5 minutes)
{ "ttl": 3600 } — custom TTL in seconds; must be positive and is clamped to 60–3600
false — bypass both reads and writes; pass this when you explicitly retry for a fresh answer

The cache key is scoped to org + model + the full request body (minus gateway-internal fields) + output schema, so changing response_format, seed, or any other parameter produces a distinct entry. A fallback-winning response is cached under the model that actually answered. The response header tells you what happened:

Header value	Meaning
`HIT`	Served from cache — no upstream call, no provider spend
`MISS`	Cached the fresh response on this request
`BYPASS`	Cache was requested but the request isn’t cacheable (streaming, temperature ≠ 0, tools), or `cache: false`

import { GatelitClient } from "@gatelit/sdk"

const response = await client.chat({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Classify this email: order placed" }],
  temperature: 0,
  cache: true,
})

if (response.meta.cacheHit) {
  console.log("served from cache — no cost")
}

Response

Non-streaming

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "model": "openai/gpt-4o",
  "choices": [{
    "index": 0,
    "message": { "role": "assistant", "content": "..." },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 8,
    "total_tokens": 23
  }
}

Response headers

Header	Description
`x-gatelit-request-id`	Unique request ID for log lookup
`x-gatelit-model`	Resolved model in `provider/model` format
`x-gatelit-auth-mode`	`Bearer`, `GatelitSigned`, or `GatelitKey`
`x-gatelit-fallback`	`"true"` when a fallback model was used
`x-gatelit-fallback-reason`	`rate_limit`, `provider_error`, `timeout`, `refusal_detected`, or `cache` (a cached response produced by a different model)
`x-gatelit-fallback-model`	The fallback model that responded
`x-gatelit-clamped`	Params the gateway clamped or stripped against the resolved model’s limits (e.g. `max_tokens=10000->4096,temperature=removed`). Only present when the gateway modified the request
`x-gatelit-cache`	`HIT`, `MISS`, or `BYPASS` when the request opted into caching

Params are forwarded to the provider as-is within the resolved model’s catalog limits. The gateway clamps out-of-range values (e.g. max_tokens above the model’s cap) rather than erroring — when that happens, x-gatelit-clamped tells you what changed. The provider enforces the budget itself: a response that ends with finish_reason: "length" hit the max_tokens cap. Streaming responses surface the same header; the final SSE chunk carries the model’s finish_reason.

Streaming

When stream: true, the response is an OpenAI-compatible SSE stream. Each data: line is a JSON chunk. Final line is data: [DONE].

Examples

curl https://gateway.gatelit.dev/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: GatelitKey glk_..." \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

import { GatelitClient } from "@gatelit/sdk"

const client = new GatelitClient({
  gatewayUrl: "https://gateway.gatelit.dev",
  getToken: async () => ({ token: "glk_...", scheme: "GatelitKey" }),
})

const response = await client.chat({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Hello!" }],
})

Structured output

curl https://gateway.gatelit.dev/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: GatelitKey glk_..." \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [{"role": "user", "content": "Alice is 30."}],
    "output_schema": {
      "mode": "json_schema",
      "type": "object",
      "properties": {
        "name": {"type": "string"},
        "age": {"type": "integer"}
      },
      "required": ["name", "age"]
    }
  }'

With fallback chain

curl https://gateway.gatelit.dev/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: GatelitKey glk_..." \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [{"role": "user", "content": "Explain TCP"}],
    "fallback_chain": [
      {"model": "anthropic/claude-3-5-haiku-20241022", "triggers": ["rate_limit"]},
      {"model": "google/gemini-2.5-flash"}
    ]
  }'

import { GatelitClient } from "@gatelit/sdk"

const client = new GatelitClient({
  gatewayUrl: "https://gateway.gatelit.dev",
  getToken: async () => ({ token: "glk_...", scheme: "GatelitKey" }),
})

const response = await client.chat({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Explain TCP" }],
  fallbackChain: [
    { model: "anthropic/claude-3-5-haiku-20241022", triggers: ["rate_limit"] },
    { model: "google/gemini-2.5-flash" },
  ],
})

// When a fallback fired, the response exposes it:
if (response.meta.usedFallback) {
  console.log(`Handled by ${response.meta.fallbackModel} (${response.meta.fallbackReason})`)
}

Trigger conditions (triggers): rate_limit, provider_error, timeout, refusal_detected. Without triggers, the fallback fires on any failure. A fallback candidate never falls through when its provider is disabled for your org — it is skipped in order.

Errors

Status	Code	Meaning
400	`gateway_error`	Invalid model format or missing fields
401	`unauthorized`	Missing, invalid, or expired token
502	`provider_error`	Upstream provider returned an error

Authorizations

Authorization

string

header

required

OIDC JWT issued by your auth provider (Alpha). Full self-serve configuration is coming soon — contact us for early access. Use this for browser/frontend clients where a user is signed in.

Body

application/json

model

string

required

Model in provider/model-id format. Supported providers: openai, anthropic, google, mistral.

Examples:

"openai/gpt-4o"

"anthropic/claude-3-5-sonnet-20241022"

"google/gemini-2.5-flash"

"mistral/mistral-large-latest"

messages

object[]

required

Minimum array length: 1

Show child attributes

max_tokens

integer

Maximum tokens to generate. The gateway maps this to the correct upstream parameter — max_completion_tokens for OpenAI reasoning models, max_tokens for everything else.

Example:

1024

temperature

number

Sampling temperature. Omit for reasoning models (o3, o3-mini, o4-mini) — the gateway strips it automatically based on the model catalog.

Required range: 0 <= x <= 2

Example:

0.7

top_p

number

Nucleus sampling. Supported by all providers.

Required range: 0 <= x <= 1

stream

boolean

default:false

When true, the response is an OpenAI-compatible SSE stream. Each data: line is a JSON object regardless of the upstream provider.

user

string

End-user identifier (standard OpenAI field). Forwarded to the provider unchanged, and used as the logged end-user ID when no signed-token claim or x-gatelit-end-user-id header is present.

Example:

"customer_42"

metadata

object

Arbitrary tags attached to the request — environment, feature, session, tenant, and so on. Values must be string, number, or boolean; keys are limited to 64 characters and string values to 128 characters. Stripped before forwarding to the provider, stored on the log entry, and filterable in the dashboard.

Show child attributes

Example:

{
  "environment": "production",
  "feature": "doc-summariser",
  "session_id": "sess_123"
}

output_schema

object

Structured output configuration. The gateway translates to the correct provider-specific format automatically.

Show child attributes

prompt_id

string

Associates this request with a saved prompt in the dashboard. The prompt's content is not used — only the ID is logged for correlation.

fallback_chain

object[]

Ordered list of fallback models to try if the primary model fails. Each entry specifies a model and the error types that trigger it.

Show child attributes

cache

Opt-in exact-match response cache. Only deterministic requests are cached — non-streaming, explicit temperature: 0, no tools — and only successful upstream responses are stored. Cache reads happen before any upstream call; a fallback-winning response is cached under the model that actually answered. The response header x-gatelit-cache reports HIT, MISS, or BYPASS.

Response

Successful response.

Non-streaming: Content-Type: application/json
Streaming: Content-Type: text/event-stream

Response headers always include:

x-gatelit-request-id — use for log lookup in the dashboard
x-gatelit-model — the provider/model that responded
x-gatelit-auth-mode — Bearer, GatelitSigned, or GatelitKey

OpenAI-compatible chat completion response. Returned when stream is false (default).

string

object

string

Example:

"chat.completion"

created

integer

model

string

The upstream model that handled the request.

choices

object[]

Show child attributes

usage

object

Show child attributes

How it works

API Reference

Chat Completions

Auth

Request

Content blocks

Gateway-specific fields

Metadata

Response caching

Response

Non-streaming

Response headers

Streaming

Examples

Structured output

With fallback chain

Errors

Authorizations

Body

Response

​Auth

​Request

​Content blocks

​Gateway-specific fields

​Metadata

​Response caching

​Response

​Non-streaming

​Response headers

​Streaming

​Examples

​Structured output

​With fallback chain

​Errors

Authorizations

Body

Response

Auth

Request

Content blocks

Gateway-specific fields

Metadata

Response caching

Response

Non-streaming

Response headers

Streaming

Examples

Structured output

With fallback chain

Errors