Create an experiment | Respan Docs

Create an experiment and start asynchronous workflow execution over a dataset.

Request

Create and asynchronously run an experiment. Provide exactly one scoring path: evaluator_ids/evaluator_slugs or evaluator_workflow_ids.

dataset_idstringRequired

Dataset ID to process.

workflowlist of objectsRequired

Workflow tasks to run for each dataset row.

evaluator_idslist of stringsOptional

Preferred evaluator identifiers for scoring. Mutually exclusive with evaluator_workflow_ids.

evaluator_slugslist of stringsOptional

Backward-compatible alias for evaluator_ids. If both are provided, evaluator_ids takes precedence.

evaluator_workflow_idslist of stringsOptional

WorkflowVersion IDs configured for eval-only scoring. Mutually exclusive with evaluator IDs/slugs.

experiment_idstringOptional

Optional client-provided experiment ID. The backend generates one when omitted.

namestringOptional

Experiment name.

descriptionstringOptional

Experiment description.

span_workflow_namestringOptionalDefaults to workflow

Root workflow span name.

enable_tracingbooleanOptionalDefaults to true

Whether to create trace logs.

batch_sizeintegerOptionalDefaults to 100

Batch size for processing.

concurrencyintegerOptionalDefaults to 15

Number of concurrent workers.

generation_methodstringOptional

Optional evaluation generation method override.

Response

Created experiment.

idstring

Experiment ID.

namestring

Experiment name.

statusstring

Experiment execution status.

created_atdatetime

descriptionstring or null

Experiment description.

datasetstring or null

Dataset ID associated with the experiment.

dataset_idstring or null

Dataset ID associated with the experiment.

dataset_namestring or null

Dataset name, when available.

workflow_countinteger

Number of workflow steps.

progressdouble

Execution progress percentage.

started_atdatetime or null

completed_atdatetime or null

tagslist of maps from strings to any

Tags attached to the experiment.

workflowlist of objects

Workflow tasks configured for the experiment.

evaluator_idslist of strings

Evaluator IDs used for scoring.

evaluator_slugslist of strings

Backward-compatible evaluator identifiers stored by the backend.

evaluator_workflow_idslist of strings

Eval-only workflow versions used for scoring.

batch_sizeinteger

concurrencyinteger

enable_tracingboolean

error_messagestring or null

Failure details when status is failed.

Errors

400

Bad Request Error

401

Unauthorized Error

404

Not Found Error

$	curl -X POST https://api.respan.ai/api/v2/experiments/ \
>	-H "Authorization: Bearer sk_live_xxxxx" \
>	-H "Content-Type: application/json" \
>	-d '{
>	"dataset_id": "ds_support_qa_2024",
>	"workflow": [
>	{
>	"type": "completion",
>	"config": {
>	"model": "gpt-4o-mini",
>	"temperature": 0.2,
>	"max_tokens": 256
>	}
>	}
>	],
>	"evaluator_ids": [
>	"eval_quality"
>	],
>	"name": "gpt-4o-mini quality run",
>	"concurrency": 15
>	}'

1	{
2	"id": "exp_20240615_001",
3	"name": "gpt-4o-mini quality run",
4	"status": "pending",
5	"created_at": "2024-06-15T09:30:00Z",
6	"description": "Evaluation run for GPT-4o-mini model on Support QA Dataset to measure response quality.",
7	"dataset": "ds_support_qa_2024",
8	"dataset_id": "ds_support_qa_2024",
9	"dataset_name": "Support QA Dataset",
10	"workflow_count": 1,
11	"progress": 0,
12	"started_at": "2024-06-15T09:30:00Z",
13	"completed_at": "2024-06-15T09:30:00Z",
14	"tags": [
15	{}
16	],
17	"workflow": [
18	{
19	"type": "completion",
20	"config": {
21	"model": "gpt-4o-mini",
22	"temperature": 0.2,
23	"max_tokens": 256
24	}
25	}
26	],
27	"evaluator_ids": [
28	"eval_quality"
29	],
30	"evaluator_slugs": [
31	"quality-assessment"
32	],
33	"evaluator_workflow_ids": [
34	"wfv_quality_2024"
35	],
36	"batch_size": 100,
37	"concurrency": 15,
38	"enable_tracing": true,
39	"error_message": ""
40	}

Headers

Request

Response

Errors