Amazon Bedrock | Respan Docs

Amazon Bedrock is a fully managed service that offers foundation models from leading AI providers through a single API. Respan gives you full observability over every Bedrock invocation, streamed response, and tool call — and gateway routing through the OpenAI-compatible Respan endpoint.

Set up Respan

Create an account at platform.respan.ai and grab an API key. For gateway, also add credits or a provider key.

Run npx @respan/cli setup to set up with your coding agent.

Example projects

Python examples

Tracing

Gateway

Setup

Install packages

$ pip install respan-ai opentelemetry-instrumentation-bedrock boto3

Set environment variables

$ export AWS_ACCESS_KEY_ID="YOUR_AWS_ACCESS_KEY_ID"
$ export AWS_SECRET_ACCESS_KEY="YOUR_AWS_SECRET_ACCESS_KEY"
$ export AWS_REGION="us-east-1"
$ export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

AWS credentials are used for Bedrock requests. RESPAN_API_KEY is used to export traces to Respan.

Initialize and run

1 import json
2 import boto3
3 from respan import Respan
4 from opentelemetry.instrumentation.bedrock import BedrockInstrumentor
5 
6 respan = Respan(instrumentations=[BedrockInstrumentor()])
7 
8 client = boto3.client("bedrock-runtime", region_name="us-east-1")
9 
10 response = client.invoke_model(
11     modelId="anthropic.claude-3-sonnet-20240229-v1:0",
12     body=json.dumps({
13         "anthropic_version": "bedrock-2023-05-31",
14         "max_tokens": 1024,
15         "messages": [{"role": "user", "content": "Say hello in three languages."}],
16     }),
17     contentType="application/json",
18 )
19 result = json.loads(response["body"].read())
20 print(result["content"][0]["text"])
21 respan.flush()

View your trace

Open the Traces page to see your auto-instrumented LLM spans.

Configuration

Parameter	Type	Default	Description
`api_key`	`str \| None`	`None`	Falls back to `RESPAN_API_KEY` env var.
`base_url`	`str \| None`	`None`	Falls back to `RESPAN_BASE_URL` env var.
`instrumentations`	`list`	`[]`	Plugin instrumentations to activate (e.g. `BedrockInstrumentor()`).
`customer_identifier`	`str \| None`	`None`	Default customer identifier for all spans.
`metadata`	`dict \| None`	`None`	Default metadata attached to all spans.
`environment`	`str \| None`	`None`	Environment tag (e.g. `"production"`).

Attributes

In Respan()

Set defaults at initialization — these apply to all spans.

1 from respan import Respan
2 from opentelemetry.instrumentation.bedrock import BedrockInstrumentor
3 
4 respan = Respan(
5     instrumentations=[BedrockInstrumentor()],
6     customer_identifier="user_123",
7     metadata={"service": "chat-api", "version": "1.0.0"},
8 )

With propagate_attributes

Override per-request using a context scope.

1 import json
2 from respan import Respan, propagate_attributes
3 from opentelemetry.instrumentation.bedrock import BedrockInstrumentor
4 
5 respan = Respan(instrumentations=[BedrockInstrumentor()])
6 
7 def handle_request(user_id: str, question: str):
8     with propagate_attributes(
9         customer_identifier=user_id,
10         thread_identifier="conv_abc_123",
11         metadata={"plan": "pro"},
12     ):
13         response = client.invoke_model(
14             modelId="anthropic.claude-3-sonnet-20240229-v1:0",
15             body=json.dumps({
16                 "anthropic_version": "bedrock-2023-05-31",
17                 "max_tokens": 1024,
18                 "messages": [{"role": "user", "content": question}],
19             }),
20             contentType="application/json",
21         )
22         result = json.loads(response["body"].read())
23         print(result["content"][0]["text"])

Attribute	Type	Description
`customer_identifier`	`str`	Identifies the end user in Respan analytics.
`thread_identifier`	`str`	Groups related messages into a conversation.
`metadata`	`dict`	Custom key-value pairs. Merged with default metadata.