AWS Bedrock (tracing)

Amazon Bedrock is a fully managed service that offers foundation models from leading AI providers through a single API. Respan gives you full observability over Bedrock invocations, streamed responses, and tool calls, plus separate gateway routing through the OpenAI-compatible Respan endpoint.

Set up Respan

Create an account at platform.respan.ai and grab an API key.

Run npx @respan/cli setup to set up with your coding agent.

Use Respan Gateway

See Amazon Bedrock gateway setup to route Bedrock model calls through the Respan gateway.

Example projects

Setup

Install packages

pip install respan-ai respan-instrumentation-aws-bedrock boto3

Set environment variables

export AWS_ACCESS_KEY_ID="YOUR_AWS_ACCESS_KEY_ID"
export AWS_SECRET_ACCESS_KEY="YOUR_AWS_SECRET_ACCESS_KEY"
export AWS_REGION="us-east-1"
export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

AWS credentials are used for direct Bedrock requests. RESPAN_API_KEY is used to export traces to Respan.

Initialize and run

import json
import boto3
from respan import Respan
from respan_instrumentation_aws_bedrock import AWSBedrockInstrumentor
respan = Respan(instrumentations=[AWSBedrockInstrumentor()])
client = boto3.client("bedrock-runtime", region_name="us-east-1")
response = client.invoke_model(
    modelId="anthropic.claude-3-sonnet-20240229-v1:0",
    body=json.dumps({
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1024,
        "messages": [{"role": "user", "content": "Say hello in three languages."}],
    }),
    contentType="application/json",
)
result = json.loads(response["body"].read())
print(result["content"][0]["text"])

View your trace

Open the Traces page to see your auto-instrumented LLM spans.

Configuration

Parameter	Type	Default	Description
`api_key`	`str \| None`	`None`	Falls back to `RESPAN_API_KEY` env var.
`base_url`	`str \| None`	`None`	Falls back to `RESPAN_BASE_URL` env var.
`instrumentations`	`list`	`[]`	Plugin instrumentations to activate, such as `AWSBedrockInstrumentor()` or `new AWSBedrockInstrumentor()`.
`sdkModule`	`object \| undefined`	`undefined`	TypeScript only. Optional AWS SDK module instance for runtimes that resolve multiple copies.
`customer_identifier`	`str \| None`	`None`	Default customer identifier for all spans.
`metadata`	`dict \| None`	`None`	Default metadata attached to all spans.
`environment`	`str \| None`	`None`	Environment tag, such as `"production"`.

Supported calls

SDK call	Traced
`InvokeModel`	Model invocation spans
`InvokeModelWithResponseStream`	Streaming invocation spans after the stream is consumed
`Converse`	Chat spans with messages, output, usage, and tool definitions
`ConverseStream`	Streaming chat spans after the stream is consumed

Attributes

In Respan()

Set defaults at initialization. These apply to all spans.

from respan import Respan
from respan_instrumentation_aws_bedrock import AWSBedrockInstrumentor
respan = Respan(
    instrumentations=[AWSBedrockInstrumentor()],
    customer_identifier="user_123",
    metadata={"service": "chat-api", "version": "1.0.0"},
)

With propagate_attributes

Override per-request using a context scope.

import json
from respan import Respan, propagate_attributes
from respan_instrumentation_aws_bedrock import AWSBedrockInstrumentor
respan = Respan(instrumentations=[AWSBedrockInstrumentor()])
def handle_request(user_id: str, question: str):
    with propagate_attributes(
        customer_identifier=user_id,
        thread_identifier="conv_abc_123",
        metadata={"plan": "pro"},
    ):
        response = client.invoke_model(
            modelId="anthropic.claude-3-sonnet-20240229-v1:0",
            body=json.dumps({
                "anthropic_version": "bedrock-2023-05-31",
                "max_tokens": 1024,
                "messages": [{"role": "user", "content": question}],
            }),
            contentType="application/json",
        )
        result = json.loads(response["body"].read())
        print(result["content"][0]["text"])

Attribute	Type	Description
`customer_identifier`	`str`	Identifies the end user in Respan analytics.
`thread_identifier`	`str`	Groups related messages into a conversation.
`metadata`	`dict`	Custom key-value pairs. Merged with default metadata.