TracingLLM SDKs

Vertex AI (tracing)

Vertex AI is Google Cloud’s managed platform for foundation models, including Gemini and third-party models. Respan captures Vertex AI generation, streaming, chat, prompt, completion, token, and workflow spans.

This tracing setup uses your Google Cloud credentials directly. To route requests through Respan instead, use the Vertex AI gateway setup.

Create an account at platform.respan.ai and grab an API key.

Run npx @respan/cli setup to set up with your coding agent.

See Vertex AI gateway setup to route calls through the Respan gateway.

Setup

1

Install packages

pip install respan-ai respan-instrumentation-vertexai google-cloud-aiplatform
2

Set environment variables

export GOOGLE_APPLICATION_CREDENTIALS="path/to/your/gcp-key.json"
export GOOGLE_CLOUD_PROJECT="your-gcp-project"
export GOOGLE_CLOUD_LOCATION="us-central1"
export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

Google Cloud credentials are used for Vertex AI requests. RESPAN_API_KEY exports traces to Respan.

3

Initialize and run

import vertexai
from vertexai.generative_models import GenerativeModel
from respan import Respan
from respan_instrumentation_vertexai import VertexAIInstrumentor
respan = Respan(instrumentations=[VertexAIInstrumentor()])
vertexai.init(project="your-gcp-project", location="us-central1")
model = GenerativeModel("gemini-2.0-flash")
response = model.generate_content("Say hello in three languages.")
print(response.text)
4

View your trace

Open the Traces page to see Vertex AI generate-content, streaming, and chat spans.

Configuration

ParameterTypeDefaultDescription
api_key / apiKeystr | None / string | undefinedRESPAN_API_KEYRespan API key used to export traces.
base_url / baseURLstr | None / string | undefinedRESPAN_BASE_URLOptional Respan trace export API URL.
instrumentationslist / RespanInstrumentation[][]Include VertexAIInstrumentor() or new VertexAIInstrumentor() to activate Vertex AI tracing.

Attributes

In Respan()

Set defaults at initialization. These apply to all spans.

from respan import Respan
from respan_instrumentation_vertexai import VertexAIInstrumentor
respan = Respan(
    instrumentations=[VertexAIInstrumentor()],
    customer_identifier="user_123",
    metadata={"service": "vertex-api", "version": "1.0.0"},
)

With propagate_attributes

Override per-request attributes using a context scope.

from respan import Respan, propagate_attributes
from respan_instrumentation_vertexai import VertexAIInstrumentor
respan = Respan(instrumentations=[VertexAIInstrumentor()])
async def handle_request(user_id: str, question: str):
    with propagate_attributes(
        customer_identifier=user_id,
        thread_identifier="conv_abc_123",
        metadata={"plan": "pro"},
    ):
        response = model.generate_content(question)
        print(response.text)
AttributeTypeDescription
customer_identifierstr / stringIdentifies the end user in Respan analytics.
thread_identifierstr / stringGroups related messages into a conversation.
metadatadict / Record<string, unknown>Custom key-value pairs.

Captured operations

  • GenerativeModel.generateContent()
  • GenerativeModel.generateContentStream()
  • ChatSession.sendMessage()
  • ChatSession.sendMessageStream()