Create run from a trace

Endpoint: POST /runs/trace

Description Starts a run that scores a stored trace with agentic metrics. Pass selected_metrics to choose metrics (and per-metric args) for the run. Trace runs have no default metric set on the trace — you must send a non-empty selected_metrics map.

Whole-trace metrics (for example plan_adh, task_comp) score the full trace — omit spans on each entry. Span-scoped metrics (for example tool_select) score one OTLP span per evaluation — include a non-empty spans list of hex span ids on that entry. The run creates one evaluation per whole-trace metric, or one per listed span for span-scoped metrics. Credit checks use that evaluation count, not only the number of distinct metrics in selected_metrics.

Per-metric arguments belong inside each selected_metrics entry (metric_args). There is no top-level metric_args on this request.

Sharing rules

The new run is always owned by you (your account is the individual owner). Sharing isn't accepted directly on this endpoint — the run inherits visibility from the trace (and its data collection).

• The run's data_collection_id is copied from the trace.
• The run's org_id is set in this order:
  1. The trace's org_id, if any.
  2. Otherwise, the org_id of the trace's data collection, if any.
  3. Otherwise, if the trace belongs to a different user and you're in an organization, your own org_id (so the trace owner can see the run via the same organization).
  4. Otherwise the run is created private.
• When the run is org-shared, its project comes from your API key when the key is project-scoped (the run is assigned to the key's project). Otherwise it follows the trace/collection project chain.
• The trace must be accessible to your API key (including project scope). An inaccessible trace returns 404.

Parameters

• Body — application/json:

{
    "trace_id": 42,
    "selected_metrics": {
        "plan_adh": { "threshold": 70, "metric_args": null },
        "tool_select": {
            "threshold": 70,
            "metric_args": {},
            "spans": ["00f067aa0ba902b7", "a1b2c3d4e5f60708"]
        }
    },
    "threshold": "0–100 | null",
    "model_slug": "string | null",
    "reasoning_level": "string | null",
    "is_blocking": false,
    "alias": "string | null"
}

trace_id

• Required. JSON integer id of the stored trace to score. A digit string ("74"), a whole-number float (1.0), or a boolean (true / false) returns 422 before lookup ("Input should be a valid integer").

selected_metrics

• Required. Non-empty map of metric shortnames to metric entries. Omitting the field or sending {} returns 422.
• Keys are metric shortnames. Metric ids are rejected. Unknown or inactive shortnames return 400. Unknown keys on a metric entry are rejected (extra: forbid).
• Whole-trace metric entry: { "threshold": 0–100 | null, "metric_args": object | null } — do not send spans.
• Span-scoped metric entry: the same fields plus required spans: a non-empty array of OTLP span ids (16-character lowercase hex, the same values as on trace span objects; an optional 0x prefix is accepted and normalized). No duplicate ids in one entry.
• threshold is optional: omit it (or send null) to use that metric's catalog default. metric_args is optional (null or omitted means the metric's argument defaults). See the metric's doc page for accepted argument names and its default threshold.
• Record-scoped metrics (single input/output) cannot be used on trace runs.

threshold (run-level)

• Optional. Integer in 0–100. When omitted (or null), the run threshold is 80. Values outside that range return 422. A digit string is still accepted for this field.

model_slug

• Optional. When omitted, the server uses the platform default OpenAI evaluation model (gpt-5.6-luna with reasoning level none, same as dataset runs). That model (or the one you pass) is used for every LLM step in the evaluation.

reasoning_level

• Optional run-level setting applied to every evaluation in the run. There is no per-metric override on trace runs — only this field applies.
• Same resolution and validation rules as Single Evaluation (reasoning_level on POST /evaluate), evaluated against the run's model_slug (or default).
• The 201 response includes reasoning_level on the run and on each item in evaluations[].

alias

• Optional display name for the run (max 255 characters). Must be unique per account.
• If omitted, the server generates one as {timestamp}_{trace_name_or_id}_{user_id}.

is_blocking

• Must be a JSON boolean (true or false). A string ("yes", "true") or a number (1, 0) returns 422. Omitting the field is false.
• false (default) — The run and evaluation records are saved, then evaluation work is started in the background; the 201 response returns right away. Results may still be missing in that payload — poll GET /runs/{run_id} until evaluations complete.
• true — The API waits until every evaluation for this run has been executed. The response body normally includes filled-in scores, aggregate_results, and finished_at, and cost totals for the run are finalized before you receive 201.

Error responses

• 401, 402 — Auth or insufficient balance (charged per evaluation created — see description above).
• 403 — API key scoping violation; org membership required when scoring another member's trace.
• 422 — Body validation failed (including a trace_id that is not a JSON integer — a digit string, a whole-number float, or a boolean — "Input should be a valid integer", before lookup; an is_blocking value that is not a JSON boolean — a string such as "yes" or "true", or a number such as 1 or 0; a run-level or per-metric threshold outside 0–100; empty selected_metrics; invalid or empty spans; malformed span id — not 8-byte hex or all zeros; duplicate span id in one entry; unknown fields on a metric entry). An empty selected_metrics map is the "at least one metric" check, separate from the trace_id type check.
• 404 — Trace not found or not accessible (Trace with id {id} not found.); Model with slug {slug} not found.; run type catalog not seeded (Run type with name TRACE not found.).
• 400 — metric id used as a map key (Invalid metric key '<id>'. Use a metric shortname, not a metric id.); unknown or inactive metric shortnames; deactivated model_slug (The following models are not active: …); invalid thresholds or metric_args; duplicate metric in selected_metrics; duplicate run alias (unique per account or org-shared alias conflict); invalid reasoning_level for the run's model (same cases as Single Evaluation). Metric and span rules (possibly combined in one detail string): record-scoped metrics — These metrics score a single input and output and cannot be run on a trace: <shortnames>.; span-scoped metric without spans — These metrics score a single span and need a spans list of OTLP span ids: <shortnames>.; whole-trace metric with spans — These metrics score the whole trace and do not take a spans list: <shortnames>.; span id not on trace — These span ids are not on this trace: <ids>.
• 500 — An error occurred while creating the trace run.

Responses

• 201 — same run object shape as GET /runs/{run_id}, with run_type: "Trace", trace_id set, dataset: null, permissions for this caller, and evaluations with record: null and record_index: null. Each evaluation may include span_id (hex string) for span-scoped work, or null for whole-trace metrics. number_of_metrics is the count of distinct metrics in selected_metrics; evaluations.length can be larger when a span-scoped metric lists multiple spans. A trace run reports can_share, can_assign_project, and can_move_collection as false — see Introduction — Permissions.

Example response (201)

{
    "id": 993,
    "user": "analyst@acme.com",
    "author_email": "analyst@acme.com",
    "run_type": "Trace",
    "run_source": "API Call",
    "trace_id": 42,
    "dataset": null,
    "data_collection": "Agent workflows",
    "org_id": null,
    "permissions": {
        "can_edit": true,
        "can_delete": true,
        "can_share": false,
        "can_assign_project": false,
        "can_move_collection": false
    },
    "number_of_metrics": 2,
    "result": null,
    "threshold": 70,
    "model_slug": "gpt-5.6-luna",
    "reasoning_level": "none",
    "alias": "20260401_support-agent_7",
    "aggregate_results": null,
    "total_cost": null,
    "started_at": "2026-04-01T10:00:00Z",
    "finished_at": null,
    "is_gte_threshold": null,
    "evaluations": []
}

curl

curl -X POST "https://api.aegisevals.ai/api/v1/runs/trace" \
  -H "Authorization: Bearer sk_00000000000000000000000000000000" \
  -H "Content-Type: application/json" \
  -d '{"trace_id":42,"selected_metrics":{"plan_adh":{"threshold":70,"metric_args":null},"task_comp":{"threshold":null,"metric_args":{}}},"is_blocking":false}'

Agentic metrics on traces

Use these shortnames in selected_metrics. Whole-trace metrics omit spans; span-scoped metrics require a spans list (OTLP hex ids from the target trace).

Whole trace

• plan_adh — Plan adherence
• plan_qual — Plan quality
• task_comp — Task completion
• traject_eff — Trajectory efficiency

Single span

• tool_select — Tool selection correctness

Span-scoped example

curl -X POST "https://api.aegisevals.ai/api/v1/runs/trace" \
  -H "Authorization: Bearer sk_00000000000000000000000000000000" \
  -H "Content-Type: application/json" \
  -d '{"trace_id":42,"selected_metrics":{"tool_select":{"threshold":70,"spans":["00f067aa0ba902b7"]}},"is_blocking":false}'