Frequently Asked Questions

Respan is an open-source AI gateway with built-in observability and evals. Self-host the gateway for free, or run it on Respan to route, trace, and evaluate every request across 1,000+ models. Backed by Y Combinator (W24) and Gradient Ventures. Processing 80+ trillion tokens for world-class founders, engineers, and product teams.

Keywords AI rebranded to Respan in 2026.

Respan is for engineering teams running LLM applications and agents in production. It works for a solo developer running one agent and for a large team with compliance requirements and multiple applications in production.

Both. Respan is an open-source AI gateway with built-in observability, evaluations, and prompt management in one product. The gateway routes requests across models and providers. Observability traces what your agents did in production. Evals score output quality. All of it reads from the same data, so a request routed through the gateway is automatically traced and can be evaluated without extra instrumentation.

Yes. Respan is SOC 2 Type II certified. Data is encrypted in transit with TLS 1.2+ and at rest with AES-256, access follows least-privilege controls, and Respan runs regular penetration testing and security audits. SOC 2 reports are available on the Team plan and above. Full policies and control documentation are in the Respan Trust Center.

Retention depends on your plan. Free retains logs and traces for 7 days, Team for 30 days, and Enterprise is custom. Team and Enterprise also support custom retention periods and conditional retention, so you can keep specific data longer than the default. Enterprise adds data retention management, PII masking, and the ability to omit logs entirely.

Respan does not currently support self-hosting, though it is on our roadmap to be added in the future.

Yes. HIPAA compliance is available as a $249/month add-on, and Respan signs a BAA with Enterprise customers.

Respan's gateway provides a single OpenAI-compatible endpoint for 1,000+ models across every major provider, including OpenAI, Anthropic, Google Gemini, and Google Vertex AI, Azure OpenAI, AWS Bedrock, Groq, Fireworks, Together AI, Perplexity, OpenRouter, Nebius, and Novita. A full list of supported models is at respan.ai/models.

The Respan gateway adds roughly 50 to 150ms of latency per request. For most applications that is negligible against model response times. If latency is critical to your application, you can skip the gateway and use Respan's tracing SDK instead, which sits outside the request path and adds no latency. Gateway traffic is logged automatically either way, so routing through the gateway means you get observability without separate instrumentation.

Yes. Respan accepts traces over OTLP, so you can send data from any OpenTelemetry-compatible SDK or collector without using a Respan-specific SDK. OpenTelemetry support is available on all plans, including Free. If you already run OpenTelemetry instrumentation, you can point your exporter at Respan and keep your existing setup. Traces can also be sent through the JSON API or the Respan SDKs.

Respan has official SDKs for Python and TypeScript/JavaScript. Framework integrations are available for OpenAI SDK, OpenAI Agents SDK, Vercel AI SDK, LangChain, LangGraph, LlamaIndex, Mastra, and Google GenAI. If you work in a language without an official SDK, you can send traces over OTLP or the JSON API from any OpenTelemetry-compatible library.

Yes. Respan's integrations help you connect to LLM providers including OpenAI, Anthropic, Google Gemini, Google Vertex AI, Azure OpenAI, AWS Bedrock, Groq, Fireworks, Together AI, Perplexity, OpenRouter, Nebius, and Novita for gateway routing, using Respan credits or your own provider keys.

Yes. The Respan MCP server gives your AI tools direct access to your Respan data, covering logs, traces, prompts, and monitors. It works with Claude, ChatGPT, Claude Code, Cursor, and Codex, and connects over HTTP at mcp.respan.ai/mcp. Enterprise customers get a dedicated endpoint.

Create an account at platform.respan.ai, then generate an API key from the API keys page. Add credits or connect your own provider key on the Integrations page. From there, pick the path that matches what you need: the gateway quickstart to route requests across models, the tracing quickstart to instrument an existing application, or the evals quickstart to start scoring output quality.

Go to platform.respan.ai, open the API keys page under settings, and create a new key. Copy it when it's shown, since keys are only displayed once. Respan API keys authenticate your requests to the gateway, the tracing SDKs, and the MCP server.

There are two ways. To try it yourself, sign up and run npx @respan/cli setup. It detects your project language, installs the SDK, instruments your code, and creates an API key, so you're looking at your own traces in a few minutes. To see it applied to your use case with the team, book a demo.

Respan has three plans. Free is $0 and includes 100k logs, 1k scores, 5 datasets, 2 evaluators, 5 prompts, and unlimited seats on one workspace. Team is $199 per month billed yearly, and adds unlimited datasets, evaluators, and prompts, 10k scores, 5 members with additional seats at $15 each, 30-day retention, a private Slack channel, and a SOC 2 report. Enterprise is custom priced and adds SAML SSO, custom SLAs, a dedicated support engineer, and volume discounts. Usage beyond plan limits is billed at $8 per 100k logs and $1 per 1k scores.

Create a route with mode fallback, list the models in the order to try, and send the route's name as model. Respan tries them in that order, moving to the next model once the current one exhausts its eligible attempts. Retries run against the current model first, then the gateway advances to the next entry, so longer chains and higher retry counts increase worst-case latency. See Fallbacks for the full execution sequence. Fallback reduces the impact of a provider failure but doesn't guarantee availability. Preflight rejections and fail-fast failures don't advance through the list, and if every eligible model fails, the gateway returns the terminal error.

Create a route with mode split, add the models you want to distribute across, set a weight for each, and send the route's name as model. You can also balance across deployments rather than models, where a deployment means one set of provider credentials, so adding two OpenAI keys gives you two deployments, each with its own weight. Load balancing only selects which model or key to try first; see Load balancing for what happens when it fails.

Respan applies limits at three scopes. Per-API-key limits are set on each key's detail page and cover a lifetime cost cap, recurring caps on cost, requests, or tokens, and an expiry date. Organization spend caps and per-customer budgets are both under the limits page, where you set a billing period, a hard spend cap, and an optional warning threshold. Customer limits cover a monthly budget and a requests-per-minute rate limit, applied per customer_identifier. Every limit either warns, which fires a notification and lets the request through, or blocks, which rejects it in real time. Blocked requests never reach the provider, so retries and fallback don't run for them.

Traces appear in Respan within seconds of the request completing. If you don't see a trace, the most common cause is a process exiting before the SDK flushes its buffer, which happens with scripts and serverless functions.

Cost and token tracking is automatic. Every span records tokens, cost, latency, and the model used, whether the request came through the gateway, the tracing SDK, or a framework integration. On the metrics dashboard you can break requests, tokens, and cost down by model, user, and API key. To track spend by feature or by customer, tag spans with customer_identifier, metadata, or custom attributes, then filter and group on those. Setup details are in the metrics dashboards docs.

Error tracking works automatically once your traffic is flowing into Respan. Failed spans are grouped into error groups by error class, provider, endpoint, and HTTP status, and Respan detects incidents on its own when error rates spike or drift above their baseline. Open the errors page to see the timeline, filter by environment, error type, provider, fault domain, or HTTP status, and drill into an incident to see contributing groups and affected models and customers. You can acknowledge incidents, add internal notes, and mark groups resolved, ignored, or reopened. Incident detection is separate from monitors, which alert you when a metric crosses a threshold you define.

A monitor is a saved rule that aggregates a metric over a time window and alerts you when it crosses a threshold. Create one at the monitors page from scratch or from a cost-spike, high-error-rate, or model-error template. The trigger reads as a sentence: when a metric of a source crosses an operator and threshold over a window. You can monitor count, errors, tokens, cost, latency, or limits, over windows from five minutes to seven days, and scope it with filters on model, provider, customer ID, prompt, API key, error type, or custom properties. Notifications go to email, Slack, Microsoft Teams, or a webhook. Send a test alert to confirm delivery, then deploy.

Behaviors detect semantic patterns in agent interactions, such as user frustration, agent laziness, jailbreak attempts, or a successful outcome. Unlike keyword or regex matching, a behavior captures meaning in context. Open Behaviors and select 'Enable behaviors' to turn them on, then use the status control in the behaviors table to choose which ones your organization tracks. Enablement is an organization-level setting and can require an admin. Respan ships six built-in behaviors: user frustration, laziness, unsafe, jailbreak, win, and escalation. You can create a custom behavior for a pattern specific to your product. Classification runs asynchronously, so a missing result means an interaction isn't classified yet, not that it passed.

Go to the metrics page and click 'New graph'. Templates are the fastest route: browse categories for models, latency, tokens, prompts, users, caches, storage, keys, providers, and scores, select the graphs you want, and save. For something a template doesn't cover, switch to 'Custom', give the chart a title, pick a metric and calculation such as P95 latency, optionally add span filters, and group the series by a dimension like model, provider, or customer ID. Check the preview, then create. Graphs can be dragged, resized, expanded, or removed, and the whole dashboard can be exported.

Go to the prompts page and click 'New prompt'. Add a title and description, then write your system and user prompts, using {{variable_name}} for dynamic content. Choose a default model and adjust settings like temperature and max output tokens in the sidebar. Add values in the Variables tab and click 'Run' to test. When it's working, 'Commit' with a message to save the version, then 'Deploy' to push it to production. Call it from your code by prompt ID, which you'll find in the Overview panel on the Prompts page.

Evals have four parts: something to test (a prompt or model), a dataset of test inputs, an evaluator that scores the outputs, and an experiment that runs them together. Build a dataset on the datasets page from a CSV or by sampling production spans and traces. Then create an evaluator on the evaluators page: start with a grader, which is the scoring unit and can be an LLM judge, a code check, or human review, then chain graders into a workflow using the visual block builder and deploy it. Finally, go to experiments, select your dataset, prompt version, and evaluators, and run it. Compare score distributions across experiments in the Analytics tab, then deploy the version that scores best.

Respan Red Team sends adversarial messages to your agent and evaluates its replies. In the platform, open the campaigns page and pick one of three connection methods: connect an agent using a Python adapter that runs alongside it, which works for private, stateful, or custom-auth agents; connect an endpoint if your agent exposes a browser-reachable OpenAI-compatible Chat Completions URL; or use the hosted ShopBot sandbox to try the workflow without touching your own systems. For the adapter route, install with pip install respan-redteam, authenticate, and run respan-redteam scan adapter.py --output report.json. Only assess agents you own or have explicit authorization to test.

Invite your team from the members page: click 'Invite member', enter their email, and assign a role. They get an email invitation. Respan has four roles: Owner has full access including billing, Admin manages members, API keys, and project settings, Member can use the platform and view logs, traces, prompts, and metrics, and Annotator is scoped to annotating and evaluating spans, which suits QA reviewers and graders. Only Owners and Admins can change roles.

Go to automations and choose a type. 'Online evals' runs an evaluator against live traffic: select a condition, select the evaluator, and set a sampling rate, starting around 0.1 for 10% of traffic keeps cost down. 'Alert' fires on aggregated metrics, for example when an average evaluation score drops below a threshold over the last hour, and sends to a webhook such as Slack or PagerDuty. Conditions are created separately under 'Conditions', where you set a type such as single log and a filter like metadata.agent = "support".

Go to the reports page, toggle the usage report active, then set a schedule by selecting which days of the week to send on and a time in your timezone. Under Metrics, choose what the digest includes (spans, errors, users, cost, tokens, latency, and Pulse) then group by API key, model, customer, or prompt, and set how many top results to show per group. Under Delivery, send by email to one or more addresses. The report covers activity since the last one and shows period-over-period changes.

Go to the webhooks page and select 'Create webhook'. Give it a name that identifies the receiving system and environment, enter the full HTTPS endpoint URL, choose the 'span' event type, and select the API key whose traffic should be forwarded. Data webhooks are scoped by event type and API key, so using a separate key per application or environment keeps the scope easy to reason about. Note that this is different from selecting a webhook inside a monitor, which is a notification destination rather than a data feed.

Start from the table you want to export, such as Spans or Traces, and use the download control in the toolbar. Set your time range and filters, choose a sampling percentage, select which columns to include, and pick CSV for tabular analysis or JSONL for one JSON record per line. Choose Recurring export instead of a one-time job when a downstream system needs a refreshed copy (recurring exports currently run every 15 minutes). Manage and review jobs on the exports page, which collects jobs started from logs, traces, users, or datasets.

Yes. Respan's status page shows current status and 90-day history for platform web, logging services, and gateway services, with separate sections for cloud and enterprise.

The fastest route for technical questions is the Respan Discord, where the team and community answer setup and debugging questions. Team plans and above get a private Slack channel with direct access to the Respan team. For account, billing, or anything you'd rather not post publicly, you can email us. Sales questions, demos, and enterprise inquiries schedule a demo. Most setup and troubleshooting questions are already answered in the docs.