Create an evaluator
Create a new evaluator for scoring LLM outputs. Specify type and score_value_type. Optionally use eval_class for pre-built templates.
Headers
Authorization
Bearer token. Use Bearer YOUR_API_KEY.
Request
This endpoint expects an object.
name
Evaluator display name.
type
Evaluator type.
Allowed values:
score_value_type
Score format.
evaluator_slug
Unique identifier. Auto-generated if not provided.
description
Evaluator description.
eval_class
Pre-built template.
Allowed values:
categorical_choices
Required for single_select or multi_select score types.
score_config
Score type configuration. For numerical/percentage: min_score, max_score. For single/multi select: choices array.
passing_conditions
Conditions for passing. Uses filter format (e.g. {"primary_score": {"operator": "gte", "value": 3}}).
llm_config
LLM automation config. Required fields: model, evaluator_definition.
code_config
Code automation config.
configurations
Legacy configuration format. Use llm_config/code_config instead for new evaluators.
Response
Created evaluator.
id
Evaluator ID.
name
type
score_value_type
evaluator_slug
description
score_config
llm_config
code_config
passing_conditions
created_at
Errors
401
Unauthorized Error
