> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://respan.ai/docs/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://respan.ai/_mcp/server.

# Together AI (tracing)

> Trace Together AI SDK chat, streaming, async, embeddings, rerank, image generation, and tool calls with Respan in Python or TypeScript.

The [Together AI SDKs](https://docs.together.ai/docs/quickstart) are official clients for Together's inference platform. `respan-instrumentation-together` and `@respan/instrumentation-together-ai` emit Respan spans for direct Together AI calls.

#### Set up Respan

1. **Sign up** - Create an account at [platform.respan.ai](https://platform.respan.ai)
2. **Create an API key** - Generate one on the [API keys page](https://platform.respan.ai/platform/api/api-keys)

#### Use Respan Gateway

See [Together AI gateway setup](/docs/gateway/together-ai) to route Together AI model calls through the Respan gateway.

#### Example projects

* [Python examples](https://github.com/respanai/respan-example-projects/tree/main/python/tracing/together)
* [TypeScript examples](https://github.com/respanai/respan-example-projects/tree/main/typescript/tracing/together-ai)

## Setup

#### Install packages

```bash Python
pip install respan-ai respan-instrumentation-together together python-dotenv
```

```bash TypeScript
npm install @respan/respan @respan/tracing @respan/instrumentation-together-ai together-ai dotenv
```

#### Set environment variables

```bash
export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
export TOGETHER_API_KEY="YOUR_TOGETHER_API_KEY"
```

Optional:

```bash
export TOGETHER_MODEL="meta-llama/Llama-3.2-3B-Instruct-Turbo"
```

`TOGETHER_API_KEY` is used by the Together AI SDK for direct provider calls. `RESPAN_API_KEY` is used by Respan for trace export.

#### Initialize and run

```python Python
import os

from dotenv import load_dotenv
from respan import Respan, workflow
from respan_instrumentation_together import TogetherInstrumentor
from together import Together

load_dotenv()

respan = Respan(
    api_key=os.environ["RESPAN_API_KEY"],
    instrumentations=[TogetherInstrumentor()],
)
client = Together(api_key=os.environ["TOGETHER_API_KEY"])


@workflow(name="together_chat_completion")
def run_chat() -> str:
    response = client.chat.completions.create(
        model=os.getenv("TOGETHER_MODEL", "meta-llama/Llama-3.2-3B-Instruct-Turbo"),
        messages=[
            {
                "role": "user",
                "content": "Reply with one concise sentence about tracing.",
            }
        ],
    )
    return response.choices[0].message.content or ""


try:
    print(run_chat())
finally:
    respan.shutdown()
```

```typescript TypeScript
import "dotenv/config";
import Together from "together-ai";
import { Respan } from "@respan/respan";
import { TogetherAIInstrumentor } from "@respan/instrumentation-together-ai";

const respan = new Respan({
  apiKey: process.env.RESPAN_API_KEY,
  baseURL: process.env.RESPAN_BASE_URL,
  instrumentations: [new TogetherAIInstrumentor()],
});
await respan.initialize();

const together = new Together({ apiKey: process.env.TOGETHER_API_KEY });

try {
  const response = await respan.withWorkflow({ name: "together_chat_completion" }, async () => {
    return together.chat.completions.create({
      model: process.env.TOGETHER_MODEL ?? "meta-llama/Llama-3.2-3B-Instruct-Turbo",
      messages: [{ role: "user", content: "Reply with one concise sentence about tracing." }],
    });
  });
  console.log(response.choices[0]?.message?.content ?? "");
} finally {
  await respan.shutdown();
}
```

#### View your trace

Open the [Traces page](https://platform.respan.ai/platform/traces) and search for the workflow name `together_chat_completion`.

## Configuration

| Parameter             | Type           | Default | Description                                                                                              |
| --------------------- | -------------- | ------- | -------------------------------------------------------------------------------------------------------- |
| `api_key`             | `str \| None`  | `None`  | Respan API key. Falls back to `RESPAN_API_KEY`.                                                          |
| `base_url`            | `str \| None`  | `None`  | Respan API base URL. Falls back to `RESPAN_BASE_URL`.                                                    |
| `instrumentations`    | `list`         | `[]`    | Plugin instrumentations to activate, such as `TogetherInstrumentor()` or `new TogetherAIInstrumentor()`. |
| `customer_identifier` | `str \| None`  | `None`  | Default customer identifier for all exported spans.                                                      |
| `metadata`            | `dict \| None` | `None`  | Default metadata attached to all exported spans.                                                         |
| `environment`         | `str \| None`  | `None`  | Environment tag, such as `"production"`.                                                                 |

## Supported calls

| SDK call                                                            | Traced                                                                 |
| ------------------------------------------------------------------- | ---------------------------------------------------------------------- |
| `client.chat.completions.create(...)`                               | Chat completion spans                                                  |
| `client.chat.completions.create(..., stream=True)` / `stream: true` | Streaming chat completion spans after the stream is consumed or closed |
| `await async_client.chat.completions.create(...)`                   | Async chat completion spans                                            |
| `client.completions.create(...)`                                    | Text completion spans                                                  |
| `client.embeddings.create(...)`                                     | Embedding spans with vector payloads summarized                        |
| `client.rerank.create(...)`                                         | Rerank spans                                                           |
| `client.images.generate(...)`                                       | Image generation spans                                                 |
| `tools=[...]` / `tool_calls` responses                              | Tool definitions and assistant tool calls                              |

## Attributes

Attach customer identifiers, thread IDs, workflow names, and metadata to Together AI calls with `propagate_attributes` or `propagateAttributes`.

```python Python
from respan import propagate_attributes

with propagate_attributes(
    customer_identifier="user_123",
    thread_identifier="conversation_456",
    trace_group_identifier="together_support_chat",
    metadata={"plan": "pro", "workflow_name": "together_support_chat"},
):
    response = client.chat.completions.create(
        model="meta-llama/Llama-3.2-3B-Instruct-Turbo",
        messages=[{"role": "user", "content": "Summarize our support policy."}],
    )
```

```typescript TypeScript
await respan.propagateAttributes(
  {
    customer_identifier: "user_123",
    thread_identifier: "conversation_456",
    trace_group_identifier: "together_support_chat",
    metadata: { plan: "pro", workflow_name: "together_support_chat" },
  },
  async () => {
    return together.chat.completions.create({
      model: "meta-llama/Llama-3.2-3B-Instruct-Turbo",
      messages: [{ role: "user", content: "Summarize our support policy." }],
    });
  }
);
```

| Attribute                | Type   | Description                                          |
| ------------------------ | ------ | ---------------------------------------------------- |
| `customer_identifier`    | `str`  | Identifies the end user in Respan analytics.         |
| `thread_identifier`      | `str`  | Groups related messages into a conversation.         |
| `trace_group_identifier` | `str`  | Groups spans by workflow name.                       |
| `metadata`               | `dict` | Custom key-value pairs merged with default metadata. |

## Examples

### Streaming chat

```python Python
stream = client.chat.completions.create(
    model="meta-llama/Llama-3.2-3B-Instruct-Turbo",
    messages=[{"role": "user", "content": "Write a short haiku about traces."}],
    stream=True,
)

for chunk in stream:
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="", flush=True)
```

```typescript TypeScript
const stream = await together.chat.completions.create({
  model: "meta-llama/Llama-3.2-3B-Instruct-Turbo",
  messages: [{ role: "user", content: "Write a short haiku about traces." }],
  stream: true,
});

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) process.stdout.write(content);
}
```

### Embeddings

```python Python
response = client.embeddings.create(
    model="BAAI/bge-base-en-v1.5",
    input=[
        "Respan traces Together AI chat calls.",
        "Embeddings should not export vector payloads.",
    ],
)
print(len(response.data))
```

```typescript TypeScript
const response = await together.embeddings.create({
  model: "BAAI/bge-base-en-v1.5",
  input: [
    "Respan traces Together AI chat calls.",
    "Embeddings should not export vector payloads.",
  ],
});
console.log(response.data.length);
```

### Tool calling

```python Python
response = client.chat.completions.create(
    model="meta-llama/Llama-3.2-3B-Instruct-Turbo",
    messages=[{"role": "user", "content": "What is the weather in Tokyo?"}],
    tools=[
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get the current weather for a city.",
                "parameters": {
                    "type": "object",
                    "properties": {"city": {"type": "string"}},
                    "required": ["city"],
                },
            },
        }
    ],
)
print(response.choices[0].message.tool_calls)
```

```typescript TypeScript
const response = await together.chat.completions.create({
  model: "meta-llama/Llama-3.2-3B-Instruct-Turbo",
  messages: [{ role: "user", content: "What is the weather in Tokyo?" }],
  tools: [
    {
      type: "function",
      function: {
        name: "get_weather",
        description: "Get the current weather for a city.",
        parameters: {
          type: "object",
          properties: { city: { type: "string" } },
          required: ["city"],
        },
      },
    },
  ],
});
console.log(response.choices[0]?.message?.tool_calls);
```