Azure OpenAI (tracing)

The Azure OpenAI TypeScript instrumentation traces calls made through the current openai package’s AzureOpenAI client. It also supports the legacy @azure/openai OpenAIClient when that package is present.

This tracing setup uses your Azure OpenAI credentials directly. To route requests through Respan instead, use the Azure OpenAI gateway setup.

Set up Respan

Create an account at platform.respan.ai and grab an API key.

Run npx @respan/cli setup to set up with your coding agent.

Use Respan Gateway

See Azure OpenAI gateway setup to route calls through the Respan gateway.

Setup

Install packages

$ npm install @respan/respan @respan/instrumentation-azure-openai openai

Set environment variables

$ export AZURE_OPENAI_API_KEY="YOUR_AZURE_OPENAI_API_KEY"
$ export AZURE_OPENAI_ENDPOINT="https://YOUR_RESOURCE.openai.azure.com"
$ export OPENAI_API_VERSION="2024-10-21"
$ export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

AZURE_OPENAI_API_KEY, AZURE_OPENAI_ENDPOINT, and OPENAI_API_VERSION are used for Azure OpenAI requests. RESPAN_API_KEY exports traces to Respan.

Initialize and run

1 import { AzureOpenAI } from "openai";
2 import * as OpenAI from "openai";
3 import { Respan } from "@respan/respan";
4 import { AzureOpenAIInstrumentor } from "@respan/instrumentation-azure-openai";
5 
6 const respan = new Respan({
7   apiKey: process.env.RESPAN_API_KEY,
8   baseURL: process.env.RESPAN_BASE_URL,
9   instrumentations: [
10     new AzureOpenAIInstrumentor({ openAIModule: OpenAI }),
11   ],
12 });
13 await respan.initialize();
14 
15 const client = new AzureOpenAI({
16   apiKey: process.env.AZURE_OPENAI_API_KEY,
17   endpoint: process.env.AZURE_OPENAI_ENDPOINT,
18   apiVersion: process.env.OPENAI_API_VERSION,
19   deployment: "gpt-4o-mini",
20 });
21 
22 const response = await client.chat.completions.create({
23   model: "gpt-4o-mini",
24   messages: [{ role: "user", content: "Say hello in one sentence." }],
25 });
26 console.log(response.choices[0]?.message?.content);
27 
28 await respan.flush();
29 await respan.shutdown();

View your trace

Open the Traces page to see Azure OpenAI chat, completion, embedding, and streaming spans.

Configuration

Parameter	Type	Default	Description
`apiKey`	`string \| undefined`	`RESPAN_API_KEY`	Respan API key used to export traces.
`baseURL`	`string \| undefined`	`RESPAN_BASE_URL`	Optional Respan trace export API URL.
`instrumentations`	`RespanInstrumentation[]`	`[]`	Include `new AzureOpenAIInstrumentor()` to activate Azure OpenAI tracing.
`openAIModule`	`object \| undefined`	dynamic import	Pass the imported `openai` module when your app imports `AzureOpenAI` before initializing Respan.
`azureOpenAIModule`	`object \| undefined`	dynamic import	Optional legacy `@azure/openai` module for `OpenAIClient` applications.
`traceContent`	`boolean`	`true`	Capture prompt and completion content when spans are exported.

Attributes

With propagateAttributes

Override per-request attributes using a context scope.

1 import { AzureOpenAI } from "openai";
2 import * as OpenAI from "openai";
3 import { Respan } from "@respan/respan";
4 import { AzureOpenAIInstrumentor } from "@respan/instrumentation-azure-openai";
5 
6 const respan = new Respan({
7   instrumentations: [
8     new AzureOpenAIInstrumentor({ openAIModule: OpenAI }),
9   ],
10 });
11 await respan.initialize();
12 
13 const client = new AzureOpenAI({
14   apiKey: process.env.AZURE_OPENAI_API_KEY,
15   endpoint: process.env.AZURE_OPENAI_ENDPOINT,
16   apiVersion: process.env.OPENAI_API_VERSION,
17   deployment: "gpt-4o-mini",
18 });
19 
20 async function handleRequest(userId: string, question: string) {
21   await respan.propagateAttributes(
22     {
23       customer_identifier: userId,
24       thread_identifier: "conv_abc_123",
25       metadata: { plan: "pro" },
26     },
27     async () => {
28       const response = await client.chat.completions.create({
29         model: "gpt-4o-mini",
30         messages: [{ role: "user", content: question }],
31       });
32       console.log(response.choices[0]?.message?.content);
33     }
34   );
35 }

Attribute	Type	Description
`customer_identifier`	`string`	Identifies the end user in Respan analytics.
`thread_identifier`	`string`	Groups related messages into a conversation.
`metadata`	`Record<string, unknown>`	Custom key-value pairs.

Captured operations

AzureOpenAI.chat.completions.create()
AzureOpenAI.completions.create()
AzureOpenAI.embeddings.create()
Streaming chat and text completion calls
Legacy @azure/openai OpenAIClient chat, completion, embedding, and streaming methods