Responses API

Send OpenAI Responses-shaped requests to:

POST https://api.respan.ai/api/responses

Respan authenticates the request, routes it to the selected provider, and records the response, actual model, usage, latency, and cost.

Choose a route

The Perplexity value applies only to POST /api/responses. It is ignored on Chat Completions and provider passthrough endpoints. Without the header, existing OpenAI routing is unchanged.

All routes require input. OpenAI and Azure OpenAI also require model. Perplexity may omit model only when it supplies preset or models.

OpenAI route

OpenAI is the default route. You may omit X-Respan-Route-Provider, or set it explicitly to openai.

The BYOK request below uses two keys:

• RESPAN_API_KEY authenticates the request to Respan.
• OPENAI_API_KEY authenticates Respan to OpenAI for this request.

1
import os
2
import requests
3
4
model = "openai/gpt-5.1"
5
6
response = requests.post(
7
    "https://api.respan.ai/api/responses",
8
    headers={
9
        "Authorization": f"Bearer {os.environ['RESPAN_API_KEY']}",
10
        "Content-Type": "application/json",
11
        "X-Respan-Route-Provider": "openai",
12
    },
13
    json={
14
        "model": model,
15
        "input": "Explain why small language models are useful.",
16
        "respan_params": {
17
            "credential_override": {
18
                model: {
19
                    "api_key": os.environ["OPENAI_API_KEY"],
20
                }
21
            }
22
        },
23
    },
24
)
25
response.raise_for_status()
26
27
data = response.json()
28
print(data["model"])
29
print(data["output"])

See OpenAI provider setup.

Azure OpenAI route

Set X-Respan-Route-Provider: azure to route the request through Azure OpenAI. Use an azure/-prefixed deployment name in both model and credential_override so the credential lookup is unambiguous.

The BYOK request requires:

• RESPAN_API_KEY for Respan authentication.
• AZURE_OPENAI_API_KEY for the Azure resource.
• AZURE_OPENAI_ENDPOINT, such as https://YOUR_RESOURCE.openai.azure.com.
• AZURE_OPENAI_API_VERSION, matching the deployment.

1
import os
2
import requests
3
4
model = "azure/gpt-4o"
5
6
response = requests.post(
7
    "https://api.respan.ai/api/responses",
8
    headers={
9
        "Authorization": f"Bearer {os.environ['RESPAN_API_KEY']}",
10
        "Content-Type": "application/json",
11
        "X-Respan-Route-Provider": "azure",
12
    },
13
    json={
14
        "model": model,
15
        "input": "Summarize the benefits of retrieval-augmented generation.",
16
        "respan_params": {
17
            "credential_override": {
18
                model: {
19
                    "api_key": os.environ["AZURE_OPENAI_API_KEY"],
20
                    "api_base": os.environ["AZURE_OPENAI_ENDPOINT"],
21
                    "api_version": os.environ["AZURE_OPENAI_API_VERSION"],
22
                }
23
            }
24
        },
25
    },
26
)
27
response.raise_for_status()
28
29
data = response.json()
30
print(data["model"])
31
print(data["output"])

See Azure OpenAI provider setup.

Perplexity Agent API route

The default BYOK request uses two keys:

• RESPAN_API_KEY authenticates the request to Respan.
• PERPLEXITY_API_KEY is passed as a request-scoped provider credential.

You can omit the request-scoped credential after adding Perplexity under Settings → Providers, or when managed Perplexity credentials are available through Respan credits.

1
import os
2
import requests
3
4
response = requests.post(
5
    "https://api.respan.ai/api/responses",
6
    headers={
7
        "Authorization": f"Bearer {os.environ['RESPAN_API_KEY']}",
8
        "Content-Type": "application/json",
9
        "X-Respan-Route-Provider": "perplexity",
10
    },
11
    json={
12
        "preset": "medium",
13
        "input": "Summarize the latest developments in small language models.",
14
        "max_steps": 1,
15
        "respan_params": {
16
            "credential_override": {
17
                "": {
18
                    "api_key": os.environ["PERPLEXITY_API_KEY"],
19
                }
20
            }
21
        },
22
    },
23
)
24
response.raise_for_status()
25
26
data = response.json()
27
print(data["model"])
28
print(data["output"])

The response model is the actual model selected by Perplexity. A preset-only request therefore does not need a model field.

Perplexity request shapes

Explicit model

Perplexity Agent API accepts provider-prefixed model names. Respan passes the name through unchanged.

1
{
2
  "model": "openai/gpt-5.1",
3
  "input": "Compare two approaches to retrieval-augmented generation."
4
}

Preset-only

Omit model and let Perplexity select one using a preset.

1
{
2
  "preset": "medium",
3
  "input": "Research the current state of browser agents.",
4
  "max_steps": 4
5
}

Fallback chain

Use models to provide models in the order Perplexity should try them.

1
{
2
  "models": [
3
    "openai/gpt-5.1",
4
    "anthropic/claude-sonnet-4.5"
5
  ],
6
  "input": "Explain the tradeoffs of each approach."
7
}

Streaming

Set stream: true to receive Responses API server-sent events. The stream ends with a response.completed event.

1
{
2
  "model": "openai/gpt-5.1",
3
  "input": "Give me a concise research summary.",
4
  "stream": true
5
}

Perplexity parameters

These extensions are accepted only when the Perplexity route header is present:

Perplexity-only parameters are discarded on non-Perplexity routes.

Web search filters

1
{
2
  "preset": "medium",
3
  "input": "Find the latest Respan documentation updates.",
4
  "tools": [
5
    {
6
      "type": "web_search",
7
      "filters": {
8
        "search_domain_filter": ["respan.ai"]
9
      }
10
    }
11
  ]
12
}

Perplexity credentials

Per-request BYOK

Pass the Perplexity key under the provider-scoped empty-string selector:

1
{
2
  "respan_params": {
3
    "credential_override": {
4
      "": {
5
        "api_key": "YOUR_PERPLEXITY_API_KEY"
6
      }
7
    }
8
  }
9
}

The Respan API key still belongs in Authorization: Bearer YOUR_RESPAN_API_KEY; it is not replaced by the Perplexity key.

Stored or managed credentials

Alternatively, add a Perplexity provider integration with an api_key. The credential is provider-scoped: an older integration’s Sonar available_models list does not restrict Agent API model or preset requests. Managed Perplexity credentials can be used when available through Respan credits.

If neither a customer-owned nor a managed Perplexity credential is available, the endpoint returns 401 and asks you to add a Perplexity API key.

See Perplexity provider setup.

Logging and billing

Every successful route returns X-Respan-Log-Id.

Perplexity cost details

For Perplexity, Respan logs:

• provider_id: perplexity
• the actual model returned by Perplexity
• provider-reported USD cost when present
• input, output, cache-read, cache-creation, and tool-call cost components

Perplexity’s reported usage.cost.total_cost is authoritative for logging and credit deduction. If the component values do not reconcile with the total, Respan records the difference under provider_reported. Missing or invalid cost data falls back to catalog token-pricing estimates.

Troubleshooting

See Create response for the endpoint reference, or open the OpenAI, Azure OpenAI, and Perplexity provider setup guides.