Skip to navigation
ObservabilityIntegrationsLLM SDKs

Groq (tracing)

The Groq Python SDK is the official client for Groq’s inference API. respan-instrumentation-groq activates the Groq OpenInference wrapper and normalizes emitted spans to the Respan tracing pipeline.

  1. Sign up - Create an account at platform.respan.ai
  2. Create an API key - Generate one on the API keys page

See Groq gateway setup to route Groq model calls through the Respan gateway.

Setup

1

Install packages

pip install respan-ai respan-instrumentation-groq groq python-dotenv
2

Set environment variables

export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
export GROQ_API_KEY="YOUR_GROQ_API_KEY"

Optional:

export RESPAN_BASE_URL="https://api.respan.ai/api"
export GROQ_MODEL="llama-3.1-8b-instant"
3

Initialize and run

import os
from dotenv import load_dotenv
from groq import Groq
from respan import Respan, workflow
from respan_instrumentation_groq import GroqInstrumentor
load_dotenv()
respan = Respan(
    api_key=os.environ["RESPAN_API_KEY"],
    base_url=os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api"),
    instrumentations=[GroqInstrumentor()],
)
client = Groq(api_key=os.environ["GROQ_API_KEY"])
@workflow(name="groq_chat_completion")
def run_chat() -> str:
    response = client.chat.completions.create(
        model=os.getenv("GROQ_MODEL", "llama-3.1-8b-instant"),
        messages=[
            {
                "role": "user",
                "content": "Reply with one concise sentence about tracing.",
            }
        ],
    )
    return response.choices[0].message.content or ""
try:
    print(run_chat())
finally:
    respan.shutdown()
4

View your trace

Open the Traces page and search for the workflow name groq_chat_completion.

Configuration

ParameterTypeDefaultDescription
**instrumentor_kwargsdict{}Extra keyword arguments forwarded to the upstream Groq OpenInference instrumentor.

Attributes

Attach customer identifiers, thread IDs, workflow names, and metadata to Groq calls with propagate_attributes.

from respan import propagate_attributes
with propagate_attributes(
    customer_identifier="user_123",
    thread_identifier="conversation_456",
    trace_group_identifier="groq_support_chat.workflow",
    metadata={"plan": "pro", "workflow_name": "groq_support_chat.workflow"},
):
    response = client.chat.completions.create(
        model="llama-3.1-8b-instant",
        messages=[{"role": "user", "content": "Summarize our support policy."}],
    )
AttributeTypeDescription
customer_identifierstrIdentifies the end user in Respan analytics.
thread_identifierstrGroups related messages into a conversation.
trace_group_identifierstrGroups spans by workflow name.
metadatadictCustom key-value pairs merged with default metadata.

Examples

Streaming

stream = client.chat.completions.create(
    model="llama-3.1-8b-instant",
    messages=[{"role": "user", "content": "Write a short haiku about tracing."}],
    stream=True,
)
for chunk in stream:
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="", flush=True)

Tool calling

response = client.chat.completions.create(
    model="llama-3.1-8b-instant",
    messages=[{"role": "user", "content": "What is the weather in Boston?"}],
    tools=[
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get the current weather for a city.",
                "parameters": {
                    "type": "object",
                    "properties": {"city": {"type": "string"}},
                    "required": ["city"],
                },
            },
        }
    ],
)
print(response.choices[0].message.tool_calls)