Together AI (gateway)

Route Together AI calls through the Respan gateway to use 250+ models from different providers. Only your RESPAN_API_KEY is needed — no separate TOGETHER_API_KEY required.

Setup

Install packages

$ pip install respan-ai opentelemetry-instrumentation-together together

Set environment variables

$ export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

No TOGETHER_API_KEY needed — the Respan gateway handles provider authentication.

Point an OpenAI-compatible client to the Respan gateway

1 import os
2 from openai import OpenAI
3 
4 client = OpenAI(
5     api_key=os.environ["RESPAN_API_KEY"],
6     base_url="https://api.respan.ai/api",
7 )
8 
9 response = client.chat.completions.create(
10     model="together_ai/meta-llama/Llama-3-70b-chat-hf",
11     messages=[{"role": "user", "content": "Say hello in three languages."}],
12 )
13 print(response.choices[0].message.content)

Switch models

Change the model parameter to use 250+ models from different providers through the same gateway.

1 response = client.chat.completions.create(model="together_ai/meta-llama/Llama-3-70b-chat-hf", messages=messages)
2 response = client.chat.completions.create(model="gpt-4.1-nano", messages=messages)
3 response = client.chat.completions.create(model="claude-sonnet-4-5-20250929", messages=messages)

See the full model list.

1	import os
2	from openai import OpenAI
3
4	client = OpenAI(
5	api_key=os.environ["RESPAN_API_KEY"],
6	base_url="https://api.respan.ai/api",
7	)
8
9	response = client.chat.completions.create(
10	model="together_ai/meta-llama/Llama-3-70b-chat-hf",
11	messages=[{"role": "user", "content": "Say hello in three languages."}],
12	)
13	print(response.choices[0].message.content)