For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
DiscordPlatform
DocumentationIntegrationsAPI referenceSDKsChangelog
DocumentationIntegrationsAPI referenceSDKsChangelog
    • Overview
  • Tracing
  • Gateway
      • OpenAI SDK
      • Instructor
      • Anthropic SDK
      • Google GenAI
      • LiteLLM
      • RubyLLM
      • Vertex AI
      • AWS Bedrock
      • Cohere
      • Groq
      • Mistral AI
      • Ollama
      • Watsonx
      • Together AI
      • Aleph Alpha
      • HuggingFace
      • Replicate
      • SageMaker
      • Respan API
  • Others
  • Migrating
    • Braintrust
    • Portkey
    • Langfuse
LogoLogo
DiscordPlatform
On this page
  • Setup
  • Switch models
GatewayLLM SDKs

Vertex AI (gateway)

Was this page helpful?
Previous

AWS Bedrock (gateway)

Next
Built with

Route Vertex AI calls through the Respan gateway to use 250+ models from different providers. Only your RESPAN_API_KEY is needed — no separate Google credentials required.

Setup

1

Install packages

$pip install respan-ai respan-instrumentation-vertexai google-cloud-aiplatform
2

Set environment variables

$export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

No Google credentials needed — the Respan gateway handles provider authentication.

3

Point an OpenAI-compatible client to the Respan gateway

1import os
2from openai import OpenAI
3
4client = OpenAI(
5 api_key=os.environ["RESPAN_API_KEY"],
6 base_url="https://api.respan.ai/api",
7)
8
9response = client.chat.completions.create(
10 model="vertex_ai/gemini-2.0-flash",
11 messages=[{"role": "user", "content": "Say hello in three languages."}],
12)
13print(response.choices[0].message.content)

Switch models

Change the model parameter to use 250+ models from different providers through the same gateway.

1response = client.chat.completions.create(model="vertex_ai/gemini-2.0-flash", messages=messages)
2response = client.chat.completions.create(model="gpt-4.1-nano", messages=messages)
3response = client.chat.completions.create(model="claude-sonnet-4-5-20250929", messages=messages)

See the full model list.