Prompt Engineering

Gatelit prompts are more than saved strings. They’re versioned templates with variables, reusable partials, structured output schemas, and fallback chains — all configured in the dashboard and editable without a deploy.

Variables

Variables let you inject dynamic content at runtime without touching the prompt in the dashboard.

Defining variables

In the prompt editor, add variables with a name, type, and whether they’re required:

Type	Example value
`string`	`"Alice"`
`number`	`42`
`boolean`	`true`

Using variables

Reference variables with {{double_braces}}:

You are a {{tone}} customer support agent.
Respond to the user in {{language}}.
---
User: {{user_message}}

When the prompt runs, the gateway replaces each {{name}} with the value passed in the request body.

Passing variables at runtime

curl .../v1/prompts/customer-support/run \
  -d '{
    "variables": {
      "tone": "friendly and professional",
      "language": "Spanish",
      "user_message": "My order hasn't arrived yet."
    }
  }'

Required variables that aren’t passed return a 400 error with the missing variable name.

Partials

Partials are reusable prompt fragments shared across multiple prompts. Like prompts, they’re versioned and published independently.

Creating a partial

Go to Partials → create a new partial with a name and content:

You must always respond in valid JSON.
Never include markdown formatting or extra text outside the JSON object.

Using a partial

Reference partials with <<partial-slug>>:

<<json-only-instructions>>

You are a data extraction assistant.
Extract the following fields from the user's message:
{{fields}}

When a prompt is published, the gateway resolves all <<references>> and bakes the current published versions into the prompt.

Why partials?

Consistency — update one partial, every prompt that uses it picks up the change
Separation of concerns — keep JSON formatting rules, brand voice, or compliance disclaimers in one place
Versioned independently — partials version separately from prompts, so you can test a partial change without republishing every prompt

Fallback chains

Fallback chains keep your prompts working when a provider is down, rate-limited, or returns an error.

Configuring a fallback chain

In the prompt editor, add fallback entries. Each entry has a model and optional triggers:

Trigger	When it fires
`rate_limit`	Provider returns 429
`provider_error`	Provider returns 5xx, or structured-output validation fails after retries
`timeout`	Upstream doesn’t respond within 5 minutes, or a stream stops producing data for 60 seconds
`refusal_detected`	Provider returns a content-level refusal (non-streaming responses)

Provider enablement is a policy gate, not a fallback trigger: a prompt referencing a disabled provider returns 403 model_not_allowed so the misconfiguration gets fixed — the chain only responds to upstream failures. Every model in the chain must support the prompt’s output format. When a structured output schema is set, fallback models that can’t produce JSON (json_object or json_schema) are skipped at request time — the editor flags them while you build the chain. Native and emulated JSON support both count; the gateway applies the right approach per model.

Chain strategy

Order matters. Put cheaper/faster models last:

Primary — openai/gpt-4o (best quality, most expensive)
First fallback (rate_limit) — another gpt-4o deployment or same model different region
Second fallback (provider_error, timeout) — anthropic/claude-3-5-sonnet-20241022
Last resort (no triggers — catches anything) — google/gemini-2.5-flash

The gateway tries each entry in order until one succeeds. If all fail, it returns a 502. The editor flags chain problems as you build: models outside the catalog, providers without a configured key, duplicate entries, and entries equal to the primary model.

Testing a chain

Use the model selector in the prompt runner’s Ready to test area: pick Main, any fallback entry, or a combination — each selected model runs against the request and the results appear as tabs, one per model. Fallback entries are run directly (the primary is never called unless Main is selected), so a broken chain entry surfaces here instead of in production.

In the response

When a fallback triggers, the response includes headers:

x-gatelit-fallback: true
x-gatelit-fallback-reason: rate_limit
x-gatelit-fallback-model: anthropic/claude-3-5-sonnet-20241022

Structured output

Enforce a JSON schema on the model’s response. Define it in the prompt editor with the visual schema builder:

{
  "type": "object",
  "properties": {
    "sentiment": { "type": "string", "enum": ["positive", "negative", "neutral"] },
    "confidence": { "type": "number", "minimum": 0, "maximum": 1 },
    "summary": { "type": "string" }
  },
  "required": ["sentiment", "confidence", "summary"]
}

The gateway translates this to each provider’s native format:

OpenAI → response_format: { type: "json_schema", json_schema: ... }
Anthropic → tool use with input_schema
Google → response_schema
Mistral → response_format: { type: "json_object" }

You write the schema once — the gateway handles the rest.

Testing prompts

The dashboard includes a playground (/playground) where you can test prompts interactively before publishing. Fill in variables, toggle streaming, and see the model’s output in real time. For automated testing, define scenarios on each prompt — test cases with expected assertions. Run evals against both the published version and a draft to compare results before publishing.

​Variables

​Defining variables

​Using variables

​Passing variables at runtime

​Partials

​Creating a partial

​Using a partial

​Why partials?

​Fallback chains

​Configuring a fallback chain

​Chain strategy

​Testing a chain

​In the response

​Structured output

​Testing prompts