Skip to main content
Gatelit sits between your application and every LLM provider you use. This page explains what happens inside the gateway for each request — from authentication through to the response your client receives.

​
The request lifecycle

When your client sends a request to Gatelit, it goes through four stages: 1. Authenticate The gateway reads the Authorization header and validates it based on the scheme:
  • GatelitSigned tokens are HMAC-signed JWTs issued by your backend. The gateway verifies the signature, checks the expiry, and extracts the user identity and any model restrictions embedded in the token.
  • Bearer tokens are OIDC JWTs from your identity provider (Clerk, Supabase Auth, or Auth0). The gateway verifies them against your provider’s public keys.
  • GatelitKey service keys are long-lived tokens stored in a fast cache. The gateway hashes the presented key and checks for a match.
If authentication fails for any reason, the gateway returns a 401 immediately — no request is forwarded to any provider. 2. Resolve the org’s provider key Once a request is authenticated, the gateway knows which organization it belongs to. It then looks up that org’s API key for the target provider (OpenAI, Anthropic, etc.). To keep latency low, resolved keys are stored in a cache so the gateway does not need to make a database round trip on every request. When you update a provider key in the dashboard, the cache is invalidated automatically. 3. Route to the provider The model field in your request uses provider/model-name format — for example, openai/gpt-4o or anthropic/claude-3-5-sonnet-20241022. Gatelit parses the provider prefix, selects the correct API endpoint and authentication method for that provider, and forwards the request using the org’s resolved API key. Your application code never needs to know which base URL Anthropic or Mistral uses, or how each provider expects to receive its auth credentials. That translation is handled entirely by the gateway. 4. Return the response and log the request The gateway passes the provider’s response back to your client in a normalized, OpenAI-compatible format. Streaming responses are proxied as server-sent events. Non-streaming responses include Gatelit-specific headers: a unique request ID and the resolved model name. After the response is sent, the gateway logs the request asynchronously. The log includes token counts, estimated cost, latency, the model that handled the request, and the authenticated identity. You can view these logs in the Gatelit dashboard.

​
Provider routing

When you set model: "openai/gpt-4o", Gatelit does three things:
  1. Parses openai as the provider and gpt-4o as the model name.
  2. Looks up the org’s OpenAI API key.
  3. Sends the request to OpenAI’s API, authenticating with that key.
If your organization hasn’t configured a key for that provider, the gateway falls back to a default key if one is available. Switching providers is as simple as changing the model string — no SDK reconfiguration, no new API keys in your app, and no provider-specific request formatting to worry about.

​
Bring your own keys (BYOK)

Each organization in Gatelit can store its own API keys for each provider. When a request comes in, the gateway uses that org’s key — not a shared key — to call the upstream provider. This means:
  • Spend and rate limits are scoped to your account with the provider.
  • Your API keys are stored encrypted and are never exposed through the gateway’s response.
  • Different organizations using the same Gatelit deployment remain fully isolated from each other.
You manage provider keys from the Gatelit dashboard under your organization’s settings.

​
The key cache

Resolving an org’s provider key on every request would add a database round trip to every LLM call. To avoid this, Gatelit caches resolved keys in a fast key-value store. The cache is populated the first time a key is used and is invalidated automatically when you update a key in the dashboard. From your perspective as a customer, the cache is invisible — it just means your requests are fast even at high throughput.

​
Saved prompts

Gatelit lets you store prompt templates in the dashboard and execute them by reference. A saved prompt has a system message, a user message template, and named variable slots. When you run a prompt, you provide the variable values and the gateway assembles the full message list, selects the configured model, and sends the request. Prompts are versioned. The gateway always executes the published version of a prompt, so you can iterate on drafts without affecting production requests. Every execution is correlated to the prompt in the request logs, making it easy to trace how a particular prompt version is performing.
You can link any client.chat() call to a saved prompt using the promptId option. This doesn’t change the request — it just tags the log entry with the prompt reference so you can group requests by prompt in the dashboard.

​
Structured output

When you request structured output, Gatelit applies the appropriate adapter for the target provider before forwarding the request. Each provider has a different mechanism for constraining model output to valid JSON — Gatelit abstracts that away so you can use json_object or json_schema mode regardless of which model you’re calling.