Respan API | Respan Docs

The Respan gateway provides an OpenAI-compatible API endpoint that gives you access to 250+ models from all major providers through a single API key and base URL. Every gateway request is automatically captured as a trace, so you get full observability without any SDK setup.

Set up Respan

Create an account at platform.respan.ai and grab an API key. For gateway, also add credits or a provider key.

Run npx @respan/cli setup to set up with your coding agent.

Gateway

The Respan API is the gateway itself. Every request is automatically traced — no SDK or instrumentation needed.

Endpoints

Endpoint	Base URL
OpenAI-compatible	`https://api.respan.ai/api/`
Anthropic proxy	`https://api.respan.ai/api/anthropic/`
Google Gemini proxy	`https://api.respan.ai/api/google/gemini`

Environment switching: Respan doesn’t support an env parameter in API calls. To switch between environments (test/production), use different API keys — one for your test environment and another for production. Manage keys in API Keys settings.

Setup

Set environment variables

$ export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

Make a request

1 import os
2 import requests
3 
4 response = requests.post(
5     "https://api.respan.ai/api/chat/completions",
6     headers={
7         "Content-Type": "application/json",
8         "Authorization": f"Bearer {os.environ['RESPAN_API_KEY']}",
9     },
10     json={
11         "model": "gpt-4.1-nano",
12         "messages": [{"role": "user", "content": "Say 'Hello World'"}],
13     },
14 )
15 print(response.json())

Verify

Open the Logs page to see your gateway requests captured as traces.

Switch models

Change the model parameter to use any supported provider through the same endpoint.

1 response = requests.post(
2     "https://api.respan.ai/api/chat/completions",
3     headers=headers,
4     json={"model": "gpt-4.1-nano", "messages": messages},
5 )
6 response = requests.post(..., json={"model": "claude-sonnet-4-5-20250929", "messages": messages})
7 response = requests.post(..., json={"model": "gemini-2.5-flash", "messages": messages})

Browse the full model list to see all available models.

OpenAI-compatible parameters

All standard OpenAI chat completion parameters are supported.

Parameter	Type	Description
`messages`	`array`	List of messages in OpenAI format (`role` + `content`).
`model`	`string`	Model to use (e.g. `gpt-4.1-nano`, `claude-sonnet-4-5-20250929`).
`stream`	`boolean`	Stream back partial progress token by token.
`temperature`	`number`	Controls randomness (0-2).
`max_tokens`	`number`	Maximum tokens to generate.
`top_p`	`number`	Nucleus sampling threshold.
`tools`	`array`	List of tools/functions the model may call.
`tool_choice`	`string\|object`	Controls tool selection.
`response_format`	`object`	Force JSON output.

Respan parameters

Pass Respan-specific parameters in the request body alongside OpenAI parameters. When using the OpenAI SDK, pass them via extra_body.

Observability

Parameter	Type	Description
`customer_identifier`	`string`	Tag to identify the user. See customer identifier.
`metadata`	`object`	Custom key-value pairs for filtering and search.
`custom_identifier`	`string`	Extra indexed tag (shows as “Custom ID” in spans).
`disable_log`	`boolean`	When `true`, only metrics are recorded.
`request_breakdown`	`boolean`	Returns a summarization of the response (tokens, cost, latency).

Reliability

Parameter	Type	Description
`fallback_models`	`array`	Backup models ranked by priority.
`load_balance_group`	`object`	Balance requests across models.
`retry_params`	`object`	Configure retries (`retry_enabled`, `num_retries`, `retry_after`).

Caching

Parameter	Type	Description
`cache_enabled`	`boolean`	Enable response caching.
`cache_ttl`	`number`	Cache time-to-live in seconds (default: 30 days).
`cache_options`	`object`	Set `cache_by_customer: true` to scope cache per customer.

Credentials

Parameter	Type	Description
`customer_credentials`	`object`	Pass your customer’s provider API keys.
`credential_override`	`object`	One-off credential overrides for specific models.
`model_name_map`	`object`	Map default model names to custom Azure deployment names.

Prompt management

Parameter	Type	Description
`prompt`	`object`	Use a Respan-managed prompt template.

1 {
2   "model": "gpt-4.1-nano",
3   "messages": [],
4   "prompt": {
5     "prompt_id": "your-prompt-id",
6     "variables": {"user_name": "Sarah"}
7   }
8 }

Response format

1 {
2   "id": "chatcmpl-e1b9665b-c354-41c5-bbe5-178bd0b69773",
3   "object": "chat.completion",
4   "created": 1761546960,
5   "model": "claude-sonnet-4-5-20250929",
6   "choices": [
7     {
8       "index": 0,
9       "finish_reason": "stop",
10       "message": {
11         "role": "assistant",
12         "content": "I'm doing well, thank you for asking!"
13       }
14     }
15   ],
16   "usage": {
17     "completion_tokens": 20,
18     "prompt_tokens": 2619,
19     "total_tokens": 2639
20   }
21 }