Haystack (tracing) | Respan Docs

Haystack is a Python framework for building production-ready LLM pipelines with retrieval, generation, routing, and evaluation components. Respan traces Haystack runs with respan-instrumentation-haystack, which activates the OpenInference Haystack instrumentor and exports pipeline, component, and LLM spans through the Respan tracing pipeline.

Set up Respan

Create an account at platform.respan.ai and grab an API key.

Run npx @respan/cli setup to set up with your coding agent.

Use Respan Gateway

See Haystack gateway setup to route this integration through the Respan gateway.

Example projects

Python examples

Setup

Install packages

$ pip install respan-ai respan-instrumentation-haystack haystack-ai

Set environment variables

$ export OPENAI_API_KEY="YOUR_OPENAI_API_KEY"
$ export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
$ export HAYSTACK_CONTENT_TRACING_ENABLED="true"

OPENAI_API_KEY is used by Haystack’s OpenAI component. RESPAN_API_KEY exports traces to Respan. HAYSTACK_CONTENT_TRACING_ENABLED lets Haystack include prompt and response content in spans.

Initialize and run

1 from haystack import Pipeline
2 from haystack.components.builders import PromptBuilder
3 from haystack.components.generators import OpenAIGenerator
4 from respan import Respan
5 from respan_instrumentation_haystack import HaystackInstrumentor
6 
7 respan = Respan(instrumentations=[HaystackInstrumentor()])
8 
9 pipeline = Pipeline()
10 pipeline.add_component(
11     "prompt_builder",
12     PromptBuilder(template="Answer the following question: {{question}}"),
13 )
14 pipeline.add_component("generator", OpenAIGenerator(model="gpt-4o-mini"))
15 pipeline.connect("prompt_builder", "generator")
16 
17 result = pipeline.run(
18     {"prompt_builder": {"question": "What is the capital of France?"}}
19 )
20 print(result["generator"]["replies"][0])
21 
22 respan.flush()

View your trace

Open the Traces page to see your pipeline run with pipeline spans, component spans, LLM calls, and inputs/outputs.

Configuration

Parameter	Type	Default	Description
`api_key`	`str \| None`	`None`	Falls back to `RESPAN_API_KEY` env var.
`base_url`	`str \| None`	`None`	Falls back to `RESPAN_BASE_URL` env var.
`instrumentations`	`list`	`[]`	Plugin instrumentations to activate, such as `HaystackInstrumentor()`.
`customer_identifier`	`str \| None`	`None`	Default customer identifier for all spans.
`metadata`	`dict \| None`	`None`	Default metadata attached to all spans.
`environment`	`str \| None`	`None`	Environment tag, such as `"production"`.

HaystackInstrumentor(**kwargs) passes keyword arguments through to the underlying OpenInference Haystack instrumentor.

Attributes

In Respan()

Set defaults at initialization. These apply to all spans.

1 from respan import Respan
2 from respan_instrumentation_haystack import HaystackInstrumentor
3 
4 respan = Respan(
5     instrumentations=[HaystackInstrumentor()],
6     customer_identifier="user_123",
7     metadata={"service": "rag-api", "version": "1.0.0"},
8 )

With propagate_attributes

Override per-request using a context scope.

1 from respan import propagate_attributes
2 
3 with propagate_attributes(
4     customer_identifier="user_123",
5     thread_identifier="conversation_abc",
6     metadata={"plan": "pro"},
7 ):
8     result = pipeline.run(
9         {"prompt_builder": {"question": "What is retrieval-augmented generation?"}}
10     )
11     print(result["generator"]["replies"][0])

Attribute	Type	Description
`customer_identifier`	`str`	Identifies the end user in Respan analytics.
`thread_identifier`	`str`	Groups related messages into a conversation.
`metadata`	`dict`	Custom key-value pairs. Merged with default metadata.

Decorators (optional)

Decorators are not required. Pipeline runs, components, and LLM calls are auto-traced by the instrumentor. Use @workflow and @task when you want to group multiple Haystack runs or add application-level steps around a pipeline.

1 from respan import Respan, task, workflow
2 from respan_instrumentation_haystack import HaystackInstrumentor
3 
4 respan = Respan(instrumentations=[HaystackInstrumentor()])
5 
6 @task(name="answer_question")
7 def answer_question(question: str) -> str:
8     result = pipeline.run({"prompt_builder": {"question": question}})
9     return result["generator"]["replies"][0]
10 
11 @workflow(name="haystack_qa")
12 def haystack_qa(question: str) -> str:
13     return answer_question(question)
14 
15 print(haystack_qa("What is the capital of France?"))
16 respan.flush()

Examples

RAG pipeline

Retrieval and generation components are captured as separate spans.

1 from haystack import Document, Pipeline
2 from haystack.components.builders import PromptBuilder
3 from haystack.components.generators import OpenAIGenerator
4 from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
5 from haystack.document_stores.in_memory import InMemoryDocumentStore
6 
7 document_store = InMemoryDocumentStore()
8 document_store.write_documents(
9     [
10         Document(content="Python was created by Guido van Rossum."),
11         Document(content="Rust is focused on safety and performance."),
12     ]
13 )
14 
15 template = """
16 Documents:
17 {% for document in documents %}
18 - {{ document.content }}
19 {% endfor %}
20 
21 Question: {{question}}
22 Answer:
23 """
24 
25 pipeline = Pipeline()
26 pipeline.add_component("retriever", InMemoryBM25Retriever(document_store=document_store))
27 pipeline.add_component("prompt_builder", PromptBuilder(template=template))
28 pipeline.add_component("generator", OpenAIGenerator(model="gpt-4o-mini"))
29 pipeline.connect("retriever.documents", "prompt_builder.documents")
30 pipeline.connect("prompt_builder", "generator")
31 
32 result = pipeline.run(
33     {
34         "retriever": {"query": "Who created Python?"},
35         "prompt_builder": {"question": "Who created Python?"},
36     }
37 )
38 print(result["generator"]["replies"][0])