Run one evaluation

Endpoint: POST /evaluate

Description
Runs a single evaluation: you pass a metric short name, optional model, optional reasoning level, and threshold, and the input fields the metric needs. The response echoes those fields and adds scores, costs, timing, and the reasoning level applied.

Optionally include external_id (string, max 255 characters) to attach your own stable identifier to the record. It is echoed in the response and stored on the underlying record for later correlation.

Sharing rules

The evaluation creates a backing run owned by you. With a project-scoped API key, that run is org-shared and assigned to the key's project automatically. With a private key (including an org member’s private key), the backing run stays personal; usage still bills the organization pool when you belong to an org — see Introduction — API key scoping.

Parameters

Body — application/json:

{
  "metric_shortname": "string",
  "model_slug": "string | null",
  "reasoning_level": "string | null",
  "threshold": "integer | null",
  "metric_args": "object | null",
  "external_id": "string | null",
  "prompt": "string | null",
  "input": "string | number | boolean | object | array | null",
  "context": "string | array | null",
  "output": "string | number | boolean | object | array | null",
  "golden_answer": "string | number | boolean | object | array | null"
}

model_slug

Optional. When omitted, the server uses gpt-5.6-luna with reasoning level none.

reasoning_level

Optional. Controls how much extended reasoning the evaluation model uses. Valid values depend on the chosen model_slug — they are model-specific strings, not a global enum (for example none, low, medium, high, xhigh on many recent GPT reasoning models; low, medium, high, max on many Claude thinking models).
When omitted or whitespace-only, the API uses the model's configured default when the model advertises reasoning levels; otherwise the effective level is null.
Supplying a level for a model that does not advertise reasoning levels returns 400 (Model '{slug}' does not support reasoning levels.).
An unsupported level returns 400 with the allowed values for that model (Invalid reasoning level '{level}' for model '{slug}'. Allowed values: [...]). Related 400 responses name the model when its default is missing or not allowed (Model '{slug}' has no default reasoning level configured. Allowed values: [...]; Model '{slug}' default reasoning level '{default}' is not in allowed values: [...]).
The response includes the reasoning level applied (string | null), which may differ from the value you sent.

Error responses

401, 402 — Authentication or insufficient balance; shared with other routes (Introduction — status codes).
403 — API key scoping violation (stale/cross-org key, scoped key on a personal resource).
404 — No metric with the given metric_shortname; Model with slug {slug} not found. when a provided model_slug is not a documented model slug (when model_slug is omitted, the default model is used).
400 — Metric is not active; auto-generated backing-run alias ({timestamp}_{metric_shortname}) already exists for your account; invalid reasoning_level for the model (model does not support reasoning; level not in that model's allowed set; invalid model default — error message names the model and allowed values when applicable); other conflicting-state errors.
500 — Error during evaluation or persistence.

Responses

201 — JSON object with the shape below.

Example response (201)

{
  "id": 1542,
  "run_id": 991,
  "model_slug": "gpt-5.6-luna",
  "reasoning_level": "none",
  "metric_args": null,
  "is_success": true,
  "is_gte_threshold": true,
  "threshold": 70,
  "external_id": "row-42",
  "prompt": "What is 2+2?",
  "input": null,
  "context": null,
  "output": "4",
  "golden_answer": "4",
  "started_at": "2026-04-01T09:10:03Z",
  "result": 100,
  "explanation": "Answer exactly matches the expected value.",
  "evaluation_cost": "0.0004",
  "finished_at": "2026-04-01T09:10:04Z",
  "metric_shortname": "ans_corr",
  "eval_metadata": null
}

{
  "id": 0,
  "run_id": 0,
  "model_slug": "string",
  "reasoning_level": "string | null",
  "metric_args": "object | null",
  "is_success": "boolean | null",
  "is_gte_threshold": "boolean | null",
  "threshold": 0,
  "external_id": "string | null",
  "prompt": "string | null",
  "input": "string | number | boolean | object | array | null",
  "context": "string | array | null",
  "output": "string | number | boolean | object | array | null",
  "golden_answer": "string | number | boolean | object | array | null",
  "started_at": "date",
  "result": "number | null",
  "explanation": "string | null",
  "evaluation_cost": "string | null",
  "finished_at": "date | null",
  "metric_shortname": "string",
  "eval_metadata": "object | null"
}

metric_args echoes the per-metric arguments that were used for this evaluation. Keys must be strings and must match argument names declared by the metric — unknown keys are rejected. Each value follows the type declared by the metric (string, boolean, integer, number, list, or object). See the metric's doc page for accepted argument names and types.

eval_metadata is a metric-specific JSON object with extra signals about how the score was produced (for example, intermediate computations or token usage). It is null when the metric does not emit metadata or when the evaluation failed before metadata could be collected.

evaluation_cost is the USD amount charged to your balance for this evaluation.

curl

curl -X POST "https://api.aegisevals.ai/api/v1/evaluate" \
  -H "Authorization: Bearer sk_00000000000000000000000000000000" \
  -H "Content-Type: application/json" \
  -d '{
    "metric_shortname": "ans_corr",
    "model_slug": "gpt-4o",
    "threshold": 70,
    "external_id": "row-42",
    "prompt": "What is 2+2?",
    "output": "4",
    "golden_answer": "4"
  }'

Error responses​

Error responses