Run evaluators on a dataset

POST

https://api.respan.ai/api/datasets/:dataset_id/eval-reports/create/

Create dataset evaluation tasks for one or more evaluators.

Authentication

AuthorizationBearer

Use your Respan API key for Respan API authentication. Enter only the Respan API key value; clients send Authorization: Bearer <RESPAN_API_KEY>. For /api/responses, provider credentials such as Perplexity, OpenAI, or Azure OpenAI go in Settings -> Providers or respan_params.credential_override in the request body, not in this authentication field.

AuthorizationBearer

Use a dashboard JWT only for dashboard-authenticated endpoints. Respan API-key endpoints use the respanApiKey auth field instead.

Path parameters

dataset_idstringRequired

Dataset ID.

Request

This endpoint expects an object.

evaluator_idslist of stringsRequired

Evaluator IDs to run against the dataset.

experiment_idstringOptional

Optional experiment to associate with the evaluation runs.

generation_methodstringOptional

Optional generation strategy. Defaults to auto.

Response

Evaluation tasks created.

created_taskslist of objects

errorsmap from strings to anyOptional

experiment_updatedbooleanOptional

experiment_evaluator_idslist of stringsOptional

Errors

400

Bad Request Error

401

Unauthorized Error

404

Not Found Error

$	curl -X POST https://api.respan.ai/api/datasets/dataset_id/eval-reports/create/ \
>	-H "Authorization: Bearer <respanApiKey>" \
>	-H "Content-Type: application/json" \
>	-d '{
>	"evaluator_ids": [
>	"eval_abc123"
>	]
>	}'

1	{
2	"created_tasks": [
3	{
4	"task_id": "task_01J6ABCDEF",
5	"name": "Run evaluator accuracy on dataset Support Conversations - July",
6	"dataset_id": "6d0b2c7e-3a6a-4c09-9c7e-1f2d9e2d3f0a",
7	"evaluator_slug": "accuracy",
8	"status": "pending",
9	"started_at": null,
10	"completed_at": null,
11	"error_message": "",
12	"evaluated_logs_count": null,
13	"score": null
14	}
15	],
16	"experiment_updated": false
17	}