IntegrationsModel Providers

Custom Provider

Set up a custom or self-hosted LLM provider with the Respan gateway.

  1. Sign up — Create an account at platform.respan.ai
  2. Create an API key — Generate one on the API keys page
  3. Add credits or a provider key — Add credits on the Credits page or connect your own provider key on the Integrations page

Add the Docs MCP to your AI coding tool to get help building with Respan. No API key needed.

1{
2  "mcpServers": {
3    "respan-docs": {
4      "url": "https://docs.respan.ai/mcp"
5    }
6  }
7}
This section is for Respan LLM gateway users.

Use Respan Gateway to route requests to your own self-hosted or custom LLM provider while keeping unified observability (logs, cost, latency, and reliability metrics) in Respan.

Prerequisites

  • A Respan API key
  • A running LLM endpoint that exposes an OpenAI-compatible API

Setup

1

Create a custom provider

Go to Settings > Integrations on platform.respan.ai and click Add Custom Provider.

Provide:

  • Provider name — A display name (e.g. “My vLLM Server”)
  • Base URL — The endpoint of your LLM server (e.g. https://my-vllm.example.com/v1)
  • API key — The authentication key for your server (if required)
2

Create a custom model

After creating the provider, add a custom model:

  • Model name — The model identifier your server expects (e.g. meta-llama/Llama-3-70b)
  • Provider — Select your custom provider from the dropdown
3

Use the model via the gateway

1from openai import OpenAI
2
3client = OpenAI(
4    api_key="YOUR_RESPAN_API_KEY",
5    base_url="https://api.respan.ai/api/",
6)
7
8response = client.chat.completions.create(
9    model="my-custom-model-name",
10    messages=[{"role": "user", "content": "Hello!"}],
11)
12print(response.choices[0].message.content)
4

Verify

Open the Logs page to see your requests routed through the custom provider.

API

You can also manage custom providers and models programmatically:

Custom Providers API

Create, list, update, and delete custom providers

Custom Models API

Create, list, update, and delete custom models