> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://respan.ai/docs/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://respan.ai/docs/_mcp/server.

# RubyLLM (gateway)

> Set up RubyLLM with the Respan gateway.

> **Note**
>
> RubyLLM does not have a Ruby-side tracing instrumentor. Route all calls through the Respan gateway to capture every request as a trace.

## Setup

#### Install RubyLLM

```bash
gem install ruby_llm
```

Or add it to your Gemfile.

```ruby
gem "ruby_llm"
```

#### Set environment variables

```bash
export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
```

No provider key needed. The Respan gateway handles provider authentication.

#### Configure RubyLLM with Respan

```ruby
RubyLLM.configure do |config|
  config.openai_api_key = ENV["RESPAN_API_KEY"]
  config.openai_api_base = "https://api.respan.ai/api"
end
```

#### Make your first request

```ruby
chat = RubyLLM.chat(model: "gpt-5.5")
response = chat.ask("Hello, world!")
puts response.content
```

#### View your trace

Open the [Traces page](https://platform.respan.ai/platform/traces) to see your gateway-routed calls with prompts, tokens, and cost.

## Switch models

Use another OpenAI model ID that Respan exposes through the same OpenAI-compatible endpoint.

```ruby
chat = RubyLLM.chat(model: "gpt-5.5")
chat = RubyLLM.chat(model: "gpt-5-mini")

response = chat.ask("Tell me about artificial intelligence")
puts response.content
```

> **Note**
>
> RubyLLM's OpenAI-compatible adapter does not provide a provider-neutral way to show Claude or Gemini model switches in this setup. Use the Respan API or OpenAI SDK gateway pages for provider-neutral Claude and Gemini examples.

See the [full model list](https://platform.respan.ai/platform/models).

## Streaming

```ruby
chat = RubyLLM.chat(model: "gpt-5.5")
chat.ask("Explain quantum computing") do |chunk|
  print chunk.content
end
```

## Responses API (reasoning, tools, and prompts)

RubyLLM's default OpenAI provider uses the Chat Completions API. OpenAI blocks reasoning together with function tools on that endpoint (`Function tools with reasoning_effort are not supported ... use /v1/responses instead`). To combine reasoning and tool calls, route through the OpenAI Responses API with the [`ruby_llm-responses_api`](https://github.com/khasinski/ruby_llm-responses_api) provider, pointed at the Respan gateway.

#### Install the provider gem

```ruby
# Gemfile
gem "ruby_llm"
gem "ruby_llm-responses_api"
```

#### Point the Responses provider at Respan

```ruby
require "ruby_llm"
require "ruby_llm-responses_api"

RubyLLM.configure do |config|
  config.openai_api_key  = ENV["RESPAN_API_KEY"]
  config.openai_api_base = "https://api.respan.ai/api"
end

# provider: :openai_responses routes to https://api.respan.ai/api/responses
chat = RubyLLM.chat(model: "gpt-5.5", provider: :openai_responses)
puts chat.ask("Hello!").content
```

> **Note**
>
> OpenAI reasoning models (gpt-5.x, o-series) only accept `temperature = 1` and reject `top_p`. Set `with_temperature(1)` on Responses chats that use these models, otherwise RubyLLM's default temperature is rejected.

### Reasoning with tool calls

Define a tool as usual, then combine `with_thinking` and `with_tools`. RubyLLM runs the tool loop for you.

```ruby
class GetOrderStatus < RubyLLM::Tool
  description "Look up an order's status, carrier, and tracking by order_id."
  param :order_id, desc: "The order ID, e.g. 'A-48217'."

  def execute(order_id:)
    { order_id:, status: "delayed", carrier: "UPS", tracking: "1Z999AA10123456784" }
  end
end

chat = RubyLLM.chat(model: "gpt-5.5", provider: :openai_responses)
              .with_temperature(1)
              .with_thinking(effort: :low)
              .with_tools(GetOrderStatus)

puts chat.ask("Where is my order A-48217?").content
```

### Prompt management

Pass a Respan-managed prompt under `respan_params` with `with_params`. Do not use a top-level `prompt` key on the Responses path: the OpenAI Responses API has its own native `prompt` parameter, so a top-level `prompt` collides with it and the request fails.

```ruby
chat = RubyLLM.chat(model: "gpt-5.5", provider: :openai_responses)
              .with_params(
                respan_params: {
                  prompt: {
                    prompt_id: "YOUR_PROMPT_ID",
                    variables: { store_name: "Lumen Electronics" },
                    schema_version: 2,
                    patch: { top_p: nil }
                  }
                }
              )

puts chat.ask("Hi, where is my order A-48217?").content
```

The `patch` drops the prompt's stored `top_p`, which reasoning models reject on the Responses API.

### Prompt, tools, and reasoning together

The Responses path forwards the prompt's stored parameters. When the prompt is used with a reasoning model, drop the stored `top_p` (which reasoning models reject) with a schema v2 `patch`.

```ruby
chat = RubyLLM.chat(model: "gpt-5.5", provider: :openai_responses)
              .with_temperature(1)          # reasoning models accept only temperature = 1
              .with_thinking(effort: :low)  # reasoning
              .with_tools(GetOrderStatus)   # tools (client-side)
              .with_params(                 # Respan-managed prompt
                respan_params: {
                  prompt: {
                    prompt_id: "YOUR_PROMPT_ID",
                    variables: { store_name: "Lumen Electronics" },
                    schema_version: 2,
                    patch: { top_p: nil }   # reasoning models reject top_p
                  }
                }
              )

puts chat.ask("Hi, where is my order A-48217? It still hasn't arrived.").content
```

> **Note**
>
> `respan_params` also accepts every other gateway field (`customer_identifier`, `metadata`, `fallback_models`, and so on). Prompt evaluation reruns stay on the Chat Completions path, but production traffic sent through the Responses API is still logged and can be evaluated as production data.

## Multi-tenancy with contexts

Use RubyLLM contexts to isolate per-tenant configuration.

```ruby
tenant_ctx = RubyLLM.context do |config|
  config.openai_api_key = tenant.respan_api_key
  config.openai_api_base = "https://api.respan.ai/api"
end

chat = tenant_ctx.chat(model: "gpt-5.5")
response = chat.ask("Hello!")
```

## Rails integration

Set your Respan config in an initializer.

```ruby
# config/initializers/ruby_llm.rb
RubyLLM.configure do |config|
  config.openai_api_key = ENV["RESPAN_API_KEY"]
  config.openai_api_base = "https://api.respan.ai/api"
end
```

Use `acts_as_chat` as normal, and all LLM calls will be routed through Respan.