JSON output

chatJson<T>() combines a chat request with automatic JSON parsing and TypeScript typing. It wraps chat() and calls JSON.parse() on the response — if the model returns anything other than valid JSON, it throws a GatelitError.

JSON object mode

Use mode: "json_object" when you need valid JSON but don’t care about the exact shape:

const result = await client.chatJson<{
  sentiment: string
  score: number
  reasoning: string
}>({
  model: "openai/gpt-4o",
  messages: [
    {
      role: "user",
      content: "Analyze this review: 'The product arrived late but works great.'",
    },
  ],
  outputSchema: { mode: "json_object" },
})

console.log(result.sentiment) // "mixed"
console.log(result.score)     // 6

JSON schema mode

Use mode: "json_schema" to enforce an exact shape. The gateway translates your schema to the provider’s native format — OpenAI structured outputs, Anthropic tool use, Google response schema, Mistral JSON mode:

const result = await client.chatJson<{
  name: string
  age: number
  city: string
}>({
  model: "anthropic/claude-3-5-sonnet-20241022",
  messages: [
    {
      role: "user",
      content: "Extract from: 'Alice, 30 years old, lives in Portland.'",
    },
  ],
  outputSchema: {
    mode: "json_schema",
    type: "object",
    properties: {
      name: { type: "string" },
      age: { type: "number" },
      city: { type: "string" },
    },
    required: ["name", "age", "city"],
  },
})

console.log(result.name) // "Alice"
console.log(result.age)  // 30
console.log(result.city) // "Portland"

The same code works across OpenAI, Anthropic, Google, and Mistral — no provider-specific format wrangling.

Handling non-JSON responses

If the model returns text that isn’t valid JSON, chatJson() throws a GatelitError with code "invalid_json":

try {
  const result = await client.chatJson({
    model: "openai/gpt-4o",
    messages: [{ role: "user", content: "Write me a poem." }],
    outputSchema: { mode: "json_object" },
  })
} catch (err) {
  if (err instanceof GatelitError && err.code === "invalid_json") {
    console.log("Model ignored the JSON instruction")
  }
}

For production use, prefer json_schema over json_object. Schema mode gives the provider more explicit guidance and produces more consistent results across models.

Streaming JSON

Streaming with structured output isn’t directly supported by chatJson(). Use stream() instead and accumulate the deltas, then parse the full result:

let fullText = ""

for await (const chunk of client.stream({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Extract entities from this text..." }],
  outputSchema: { mode: "json_schema", type: "object", properties: {...}, required: [...] },
})) {
  fullText += chunk.delta
}

const parsed = JSON.parse(fullText)

The gateway still enforces the schema on the provider-side, so the accumulated response should be valid JSON matching your schema.

TypeScript

TypeScript Reference

Vue

JSON object mode

JSON schema mode

Handling non-JSON responses

Streaming JSON

​JSON object mode

​JSON schema mode

​Handling non-JSON responses

​Streaming JSON

JSON object mode

JSON schema mode

Handling non-JSON responses

Streaming JSON