Vertex AI

Overview

respan-instrumentation-vertexai captures calls made through the Google Vertex AI Python SDK and sends them to Respan through the unified OTEL pipeline.

The package patches vertexai.generative_models.GenerativeModel and ChatSession methods, including sync calls, async calls, and streaming responses. It emits canonical gen_ai.*, llm.*, and traceloop.entity.* attributes for Respan chat logs.

$ pip install respan-instrumentation-vertexai google-cloud-aiplatform

Version: 0.1.0 | Python: >=3.11, <3.14

Dependencies

Package	Version
`respan-tracing`	^2.17.0
`respan-sdk`	>=2.6.1
`google-cloud-aiplatform`	>=1.71.0
`opentelemetry-semantic-conventions-ai`	>=0.4.1

Quick start

1 import vertexai
2 from respan import Respan
3 from respan_instrumentation_vertexai import VertexAIInstrumentor
4 from vertexai.generative_models import GenerativeModel
5 
6 respan = Respan(instrumentations=[VertexAIInstrumentor()])
7 
8 vertexai.init(project="your-gcp-project", location="us-central1")
9 
10 model = GenerativeModel("gemini-2.0-flash")
11 response = model.generate_content("Say hello in three languages.")
12 print(response.text)
13 respan.flush()

Public API

VertexAIInstrumentor

The main instrumentor class. Implements the Instrumentation protocol.

1 from respan_instrumentation_vertexai import VertexAIInstrumentor

Attribute/Method	Type	Description
`name`	`str`	`"vertexai"` - unique plugin identifier.
`activate()`	`() -> None`	Patches Vertex AI generation and chat methods.
`deactivate()`	`() -> None`	Restores the original Vertex AI methods.

Captured calls

Vertex AI operation	Log Type	Description
`GenerativeModel.generate_content()`	`chat`	Sync generation calls, including streamed iterators.
`GenerativeModel.generate_content_async()`	`chat`	Async generation calls, including async streams.
`ChatSession.send_message()`	`chat`	Sync chat turns, including streamed iterators.
`ChatSession.send_message_async()`	`chat`	Async chat turns.

Span attributes

LLM spans include:

Attribute	Description
`respan.entity.log_type`	`chat` for Vertex AI model calls.
`gen_ai.system`	`google`.
`gen_ai.request.model`	Vertex AI model name.
`llm.request.type`	`chat`.
`llm.request.functions`	JSON-serialized function declarations when tools are provided.
`gen_ai.prompt.N.role` / `gen_ai.prompt.N.content`	Prompt roles and message content.
`gen_ai.completion.0.role` / `gen_ai.completion.0.content`	Assistant response role and content.
`gen_ai.completion.0.tool_calls`	JSON-serialized model function calls when returned.
`gen_ai.usage.input_tokens` / `gen_ai.usage.output_tokens`	Modern token usage fields.
`gen_ai.usage.prompt_tokens` / `gen_ai.usage.completion_tokens`	Legacy token usage aliases emitted for compatibility.
`llm.usage.total_tokens`	Total token count when provided by Vertex AI.

With propagated attributes

Use propagate_attributes() to attach per-request metadata to all spans:

1 from respan import Respan, propagate_attributes
2 from respan_instrumentation_vertexai import VertexAIInstrumentor
3 
4 respan = Respan(instrumentations=[VertexAIInstrumentor()])
5 
6 with propagate_attributes(
7     customer_identifier="user_123",
8     trace_group_identifier="vertexai_support_flow",
9     metadata={"source": "support"},
10 ):
11     response = model.generate_content("Summarize this support ticket.")

Internal modules

These are implementation details, not part of the public API:

Module	Description
`_instrumentation.py`	`VertexAIInstrumentor` class and method patching.
`_translator.py`	Vertex AI payload normalization helpers.
`_otel_emitter.py`	Synthetic OTEL span construction and injection.
`_constants.py`	Vertex-AI-specific raw field and method names.