Prompt Management

Prompt management lets you define your AI prompts in the Gatelit dashboard rather than hardcoding them in your application. You can iterate on wording, publish new versions, and run prompts by ID from any client — without a code deploy.

Creating a prompt in the dashboard

Navigate to Prompts in the Gatelit dashboard and click New Prompt. Each prompt has:

System message — optional context set before the conversation
User message template — the prompt template sent as the user turn
Model — the default provider/model-name to use when running the prompt
Temperature and max tokens — default inference parameters

Variables

Use {{variable_name}} syntax anywhere in your system or user message templates to define dynamic placeholders. Variable names must start with a letter or underscore and contain only letters, digits, and underscores.

Summarize the following article in {{num_sentences}} sentences.

Article:
{{article_text}}

When you run this prompt, you supply the variable values at runtime.

Prompt partials

Partials are reusable text blocks you can embed in any prompt template using {{> partial_name}} syntax:

{{> company_tone_guidelines}}

Summarize the following article in {{num_sentences}} sentences.

Create and manage partials under Prompts → Partials in the dashboard. Partials are resolved by the gateway before variable substitution, so a partial can itself contain {{variable}} references.

Publishing versions

Every change to a prompt creates a new version. To make a version active, click Publish on that version. The gateway always executes the published version of a prompt — unpublished draft versions are never run.

Running a saved prompt via the API

Send a POST request to /v1/prompts/:id/run with your prompt’s ID (visible in the dashboard) and any variable values:

curl -X POST https://gateway.yourapp.com/v1/prompts/a1b2c3d4-.../run \
  -H "Authorization: GatelitSigned <your-token>" \
  -H "Content-Type: application/json" \
  -d '{
    "variables": {
      "topic": "AI safety",
      "num_sentences": 3
    },
    "stream": false,
    "overrides": {
      "model": "openai/gpt-4o",
      "temperature": 0.7,
      "max_tokens": 500
    }
  }'

The request body shape:

{
  /** Variable values to substitute into the prompt template */
  variables?: Record<string, string | number | boolean | null | object>
  /** Stream the response. Default: false */
  stream?: boolean
  /** Override the prompt's saved defaults at runtime */
  overrides?: {
    /** "provider/model-name" format — overrides the saved model */
    model?: string
    temperature?: number
    max_tokens?: number
  }
}

The gateway fetches the published version of the prompt, substitutes your variables, applies any overrides, and routes the assembled request through the normal proxy pipeline — with logging and provider routing applied as usual.

Linking a request to a saved prompt

If you call client.chat() directly (rather than the /v1/prompts/:id/run endpoint), you can link the request to a saved prompt for log correlation by passing promptId:

const response = await client.chat({
  model: "openai/gpt-4o-mini",
  messages: [
    { role: "system", content: "You are a helpful summarization assistant." },
    { role: "user", content: `Summarize: ${articleText}` },
  ],
  promptId: "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
})

Requests tagged with a promptId are grouped under that prompt in the Gatelit dashboard logs, making it easy to track usage and cost per prompt.

Prompt versioning

Each time you save a prompt in the dashboard, a new version is created with an auto-incremented version number. Versions are immutable once created. To make changes, save a new version and then publish it. The gateway always runs the published version. If no version has been published yet, the /v1/prompts/:id/run endpoint returns a 404 error.

The gateway caches the published version of each prompt to minimize latency. After you publish a new version, the cache is automatically invalidated so the updated prompt takes effect immediately.

Fallback model chains

A prompt version can include a fallback model chain — an ordered list of alternative models to try if the primary model fails. Configure fallback chains on the version editor in the dashboard. Each entry in the chain specifies:

Model — the fallback model in provider/model-name format
Triggers — which error conditions activate this fallback:
- rate_limit — the provider returned a 429
- provider_error — the provider returned a 5xx
- timeout — the upstream request timed out
- context_limit — the model’s context window was exceeded

Example chain: try Claude first, fall back to GPT-4o on rate limits, and fall back to Gemini on any provider error:

Order	Model	Triggers
1	`anthropic/claude-3-5-sonnet-20241022`	(primary)
2	`openai/gpt-4o`	`rate_limit`
3	`google/gemini-2.0-flash-001`	`rate_limit`, `provider_error`

The gateway tries each model in order, stopping at the first successful response. The response.meta.model field tells you which model actually handled the request.

Get Started

Authentication

Guides

Dashboard

Prompt Management

Creating a prompt in the dashboard

Variables

Prompt partials

Publishing versions

Running a saved prompt via the API

Linking a request to a saved prompt

Prompt versioning

Fallback model chains

​Creating a prompt in the dashboard

​Variables

​Prompt partials

​Publishing versions

​Running a saved prompt via the API

​Linking a request to a saved prompt

​Prompt versioning

​Fallback model chains

Creating a prompt in the dashboard

Variables

Prompt partials

Publishing versions

Running a saved prompt via the API

Linking a request to a saved prompt

Prompt versioning

Fallback model chains