OpenAI SDK (gateway) | Respan Docs

Route OpenAI SDK calls through the Respan gateway to use 1000+ models from different providers. Only your RESPAN_API_KEY is needed — no separate provider keys required.

Setup

Install packages

pip install openai

Set environment variables

export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

No OPENAI_API_KEY needed — the Respan gateway handles provider authentication.

Point the OpenAI client to the Respan gateway

import os
from openai import OpenAI
client = OpenAI(
    api_key=os.getenv("RESPAN_API_KEY"),
    base_url="https://api.respan.ai/api",
)
response = client.chat.completions.create(
    model="gpt-5.5",
    messages=[{"role": "user", "content": "Say hello in three languages."}],
)
print(response.choices[0].message.content)

Switch models

Change the model parameter to use 1000+ models from different providers through the same gateway.

response = client.chat.completions.create(model="gpt-5.5", messages=messages)
response = client.chat.completions.create(model="claude-sonnet-4-5-20250929", messages=messages)
response = client.chat.completions.create(model="gemini/gemini-3.5-flash", messages=messages)

See the full model list.

Prompt management

Use Respan prompt management to serve prompt templates from the platform. Use schema_version: 2 for all new integrations.

Chat Completions

Prompt messages are the base layer (system/context). Body messages are appended as runtime user turns.

PROMPT_ID = "YOUR_PROMPT_ID"
response = client.chat.completions.create(
    model="gpt-5.5",
    messages=[
        {"role": "user", "content": "Add dark mode support to the dashboard"},
    ],
    extra_body={
        "prompt": {
            "prompt_id": PROMPT_ID,
            "schema_version": 2,
            "variables": {
                "feature_request": "Add dark mode support",
            },
        }
    },
)
print(response.choices[0].message.content)

Responses API

Without a route header, /api/responses uses the default OpenAI route and requires an OpenAI provider credential. You can instead route the same OpenAI-shaped request to Azure OpenAI or Perplexity Agent API with X-Respan-Route-Provider.

For the Responses API, pass prompt config under respan_params. The prompt template becomes instructions and body input is preserved.

For Perplexity Agent API, set X-Respan-Route-Provider: perplexity in extra_headers. The opt-in is header-only; putting route_provider_override: perplexity in respan_params does not activate it. See the Responses API guide.

PROMPT_ID = "YOUR_PROMPT_ID"
response = client.responses.create(
    model="gpt-5.5",
    input=[
        {"role": "user", "content": "Add dark mode support to the dashboard"},
    ],
    extra_body={
        "respan_params": {
            "prompt": {
                "prompt_id": PROMPT_ID,
                "schema_version": 2,
                "variables": {
                    "feature_request": "Add dark mode support",
                },
                "patch": {"top_p": None},
            }
        }
    },
)
print(response.output_text)

The patch drops the prompt’s stored top_p. Reasoning models reject top_p on the Responses API, so leave the patch in place unless your prompt version was saved without it.

Respan parameters

Pass additional Respan parameters via extra_body for gateway features.

response = client.chat.completions.create(
    model="gpt-5.5",
    messages=[{"role": "user", "content": "Hello"}],
    extra_body={
        "customer_identifier": "user_123",
        "fallback_models": ["gpt-5-mini"],
        "metadata": {"session_id": "abc123"},
        "thread_identifier": "conversation_456",
    },
)

See Respan params & metadata for the full list.