Create an experiment | Respan Docs

curl -X POST https://api.respan.ai/api/v2/experiments/ \
     -H "Authorization: Bearer <respanApiKey>" \
     -H "Content-Type: application/json" \
     -d '{
  "dataset_id": "ds_support_qa_2024",
  "workflow": [
    {
      "type": "completion",
      "config": {
        "model": "gpt-4o-mini",
        "temperature": 0.2,
        "max_tokens": 256
      }
    }
  ],
  "evaluator_ids": [
    "eval_quality"
  ],
  "name": "gpt-4o-mini quality run",
  "concurrency": 15
}'

{
  "id": "exp_20240615_001",
  "name": "gpt-4o-mini quality run",
  "status": "pending",
  "created_at": "2024-06-15T09:30:00Z",
  "description": "Evaluation run for GPT-4o-mini model on Support QA Dataset to assess response quality.",
  "dataset": "ds_support_qa_2024",
  "dataset_id": "ds_support_qa_2024",
  "dataset_name": "Support QA Dataset",
  "workflow_count": 1,
  "progress": 0,
  "started_at": "2024-06-15T09:30:00Z",
  "completed_at": "2024-06-15T09:30:00Z",
  "tags": [
    {}
  ],
  "workflow": [
    {
      "type": "completion",
      "config": {
        "model": "gpt-4o-mini",
        "temperature": 0.2,
        "max_tokens": 256
      }
    }
  ],
  "evaluator_ids": [
    "eval_quality"
  ],
  "evaluator_slugs": [
    "eval_quality"
  ],
  "evaluator_workflow_ids": [
    "wfv_quality_2024"
  ],
  "batch_size": 100,
  "concurrency": 15,
  "enable_tracing": true,
  "error_message": ""
}

Create an experiment and start asynchronous workflow execution over a dataset.

Authentication

AuthorizationBearer

Use your Respan API key for Respan API authentication. Enter only the Respan API key value; clients send Authorization: Bearer <RESPAN_API_KEY>. For /api/responses, OpenAI or Azure OpenAI provider credentials go in Settings -> Providers or the request body credential_override field, not in this auth field.

AuthorizationBearer

Use a dashboard JWT only for dashboard-authenticated endpoints. Respan API-key endpoints use the respanApiKey auth field instead.

Request

Create and asynchronously run an experiment. Provide exactly one scoring path: evaluator_ids/evaluator_slugs or evaluator_workflow_ids.

dataset_idstringRequired

Dataset ID to process.

workflowlist of objectsRequired

Workflow tasks to run for each dataset row.

evaluator_idslist of stringsOptional

Preferred evaluator identifiers for scoring. Mutually exclusive with evaluator_workflow_ids.

evaluator_slugslist of stringsOptional

Backward-compatible alias for evaluator_ids. If both are provided, evaluator_ids takes precedence.

evaluator_workflow_idslist of stringsOptional

WorkflowVersion IDs configured for eval-only scoring. Mutually exclusive with evaluator IDs/slugs.

experiment_idstringOptional

Optional client-provided experiment ID. The backend generates one when omitted.

namestringOptional

Experiment name.

descriptionstringOptional

Experiment description.

span_workflow_namestringOptionalDefaults to workflow

Root workflow span name.

enable_tracingbooleanOptionalDefaults to true

Whether to create trace logs.

batch_sizeintegerOptionalDefaults to 100

Batch size for processing.

concurrencyintegerOptionalDefaults to 15

Number of concurrent workers.

generation_methodstringOptional

Optional evaluation generation method override.

Response

Created experiment.

idstring

Experiment ID.

namestring

Experiment name.

statusstring

Experiment execution status.

created_atdatetime

descriptionstring or null

Experiment description.

datasetstring or null

Dataset ID associated with the experiment.

dataset_idstring or null

Dataset ID associated with the experiment.

dataset_namestring or null

Dataset name, when available.

workflow_countinteger

Number of workflow steps.

progressdouble

Execution progress percentage.

started_atdatetime or null

completed_atdatetime or null

tagslist of maps from strings to any

Tags attached to the experiment.

workflowlist of objects

Workflow tasks configured for the experiment.

evaluator_idslist of strings

Evaluator IDs used for scoring.

evaluator_slugslist of strings

Backward-compatible evaluator identifiers stored by the backend.

evaluator_workflow_idslist of strings

Eval-only workflow versions used for scoring.

batch_sizeinteger

concurrencyinteger

enable_tracingboolean

error_messagestring or null

Failure details when status is failed.

Errors

400

Bad Request Error

401

Unauthorized Error

404

Not Found Error