> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://respan.ai/docs/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://respan.ai/docs/_mcp/server.

# Groq (gateway)

> Set up Groq model calls with the Respan gateway.

Route Groq model calls through the Respan gateway to use Respan credentials, request logs, routing, fallbacks, and metadata. For direct Groq SDK tracing, see [Groq tracing setup](/docs/integrations/groq-tracing).

## Setup

#### Install packages

```bash
pip install openai python-dotenv
```

#### Set environment variables

```bash
export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
```

No `GROQ_API_KEY` is required for gateway calls when your Groq provider credentials are configured in Respan.

#### Point an OpenAI-compatible client to the Respan gateway

```python
import os

from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()

client = OpenAI(
    api_key=os.environ["RESPAN_API_KEY"],
    base_url=os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api"),
)

response = client.chat.completions.create(
    model="groq/qwen/qwen3.8-27b",
    messages=[{"role": "user", "content": "Say hello in three languages."}],
)
print(response.choices[0].message.content)
```

## Switch models

Change the `model` parameter to use 1000+ models from different providers through the same gateway.

```python
response = client.chat.completions.create(model="groq/qwen/qwen3.8-27b", messages=messages)
response = client.chat.completions.create(model="openai/gpt-6-sol", messages=messages)
response = client.chat.completions.create(model="anthropic/claude-opus-5-5", messages=messages)
```

See the [full model list](https://platform.respan.ai/platform/models).

## Respan parameters

Pass additional Respan parameters via `extra_body` for gateway features.

```python
response = client.chat.completions.create(
    model="groq/qwen/qwen3.8-27b",
    messages=[{"role": "user", "content": "Hello"}],
    extra_body={
        "customer_identifier": "user_123",
        "fallback_models": ["openai/gpt-6-sol"],
        "metadata": {"session_id": "abc123"},
        "thread_identifier": "conversation_456",
    },
)
```

See [Respan params & metadata](/docs/documentation/features/gateway/respan-params) for the full list.