Skip to main content

Introduction

The Aegis API Server is the HTTP API for programmatic access to Aegis. This section is organized into Introduction, Evaluations, and Data so you can quickly find endpoint behavior, request payloads, and response examples.

Base URL​

All documented routes live under a single prefix:

https://api.aegisevals.ai/api/v1

Core concepts​

Before calling endpoints, these are the key terms used across the API:

  • Evaluation: one scoring task where Aegis checks model output against one or more metrics (for example: correctness, safety, formatting).
  • Run: a stored execution of an evaluation job. A run usually contains many evaluations and their metric scores.
  • Dataset: uploaded CSV data that Aegis stores as rows, with chosen metrics and thresholds, so you can run evaluations repeatedly without resending the file.
  • Dataset type: a lookup that classifies a dataset (for example custom user uploads versus proprietary catalog data). Types have integer ids used when filtering dataset lists.
  • Model (catalog): a registered LLM in Aegis with id, slug, name, and a supplier. See Models for the list of valid slug values to use in runs and evaluations.
  • Dataset run: a run created from a saved dataset that already exists in Aegis.
  • Custom run: a run created by sending rows directly in the request body, without needing a pre-saved dataset.
  • Data collection: a container that groups datasets and related runs so teams can organize and review evaluation work in one place.
  • External ID (external_id): an optional string (max 255 characters) on each record that lets you attach your own stable identifier — for example a ticket id, database primary key, or row key from an upstream system. Use it to correlate Aegis evaluation results back to source data. On custom runs and single evaluations, pass external_id in the request body per row. On datasets, include an external_id CSV column (or map one via column_mappings). Dataset runs inherit external_id from the stored records; it is echoed on run and evaluation responses and included in CSV exports when present.
  • Project (project_id): organizations group org-shared resources into projects. Any resource with org_id set must also belong to a project — when you newly share (private → org-visible), include project_id in the same request or the API returns 400. With a project-scoped key (organization authorization), you can also share a private resource by sending project_id alone (without org_id) on update, or rely on the key's project on POST /datasets / POST /runs/custom at create time; the server sets org_id to your organization automatically. A private API key cannot org-share this way (personal visibility only). project_id: null is rejected while a resource stays org-shared ("Organization-shared resources must remain assigned to a project"). Unsharing (org_id: null) clears project assignment for that resource and its org-shared children. Data collections are the project anchor for their member datasets and runs — assign the collection to a project (not members directly; direct assignment returns 400). Linking to an org-shared collection inherits its project. Project-scoped API keys lock every route to one project — see API key scoping. Responses on datasets and data collections include project_id when you are entitled to see it; GET /runs/{run_id} does not. List projects with GET /projects/me (empty for private keys) or use default_project_id from GET /organizations/me (null for private keys).
  • Organization-only resource: a row with no individual owner (user / user_id is null) and an org_id set — the organization owns it (typically via admin flows). Organization-only rows cannot be unshared or moved to another organization through the public API; deleting them usually requires an organization administrator. Endpoints that create resources via API keys still attach you as the individual owner unless documented otherwise.

Endpoint guide​

Evaluate​

POST /evaluate (Single Evaluation)

  • Runs an evaluation request and returns the computed scoring output.
  • Use this for quick or direct evaluation execution from your app/backend.

Runs​

POST /runs/dataset (Create run from dataset)

  • Starts a new run using a dataset already stored in Aegis.
  • Use this for repeatable evaluations over curated data.

POST /runs/custom (Create custom run)

  • Starts a new run by sending metric config and rows directly in the request.
  • Use this when data is generated on the fly and not saved as a dataset first.

GET /runs/{run_id} (Get run)

  • Retrieves one existing run with its summary and row-level evaluation data.
  • Use this to get information about a certain run.

PUT /runs/{run_id} (Update run)

  • Updates run metadata (alias, org sharing, data collection link, project assignment for custom runs).
  • Use this when run attributes need to change after creation.

GET /runs/{run_id}/download (Download run)

  • Exports run results as CSV for analysis.
  • Use this for reporting, sharing, and offline analysis.

DELETE /runs/{run_id} (Delete run)

  • Permanently removes a run you own, or an org-only run when you are an org administrator.
  • Use this to clean up completed or obsolete evaluation runs.

Dataset types​

GET /dataset-types + GET /dataset-types/{dataset_type_id} (Get dataset types)

  • Lists all dataset types or returns one type by id (name, label, description).
  • Use this to discover type ids for dataset_type_id on Get datasets.

Models​

Models (reference) — the supported LLM slugs, display names, and flags (thinking, latest) are listed in the docs; the API server does not expose a models HTTP endpoint. Use a documented slug in model_slug when you create runs or evaluations.

Datasets​

POST /datasets (Create dataset)

  • Uploads a CSV and creates a custom dataset with metric thresholds.
  • Use this to persist evaluation data for dataset runs and data collections.

GET /datasets/all-partial, GET /datasets, GET /datasets/{dataset_id} (Get datasets)

  • Lists dataset ids, names, descriptions, paginated summaries, or one full dataset (records, runs, evaluations).
  • Use this to discover datasets and inspect stored rows.

PUT /datasets/{dataset_id} (Update dataset)

  • Updates name, description, metrics, column mappings, data collection membership, org sharing, or project assignment.
  • Use this to keep dataset information accurate over time.

DELETE /datasets/{dataset_id} (Delete dataset)

  • Deletes a custom dataset you own.

GET /datasets/{dataset_id}/download (Download dataset)

  • Exports dataset rows as CSV.

Data collections​

POST /data-collections (Create data collection)

  • Creates a new data collection container.
  • Use this when grouping datasets or runs is necessary.

GET /data-collections + GET /data-collections/{data_collection_id} (Get data collections)

  • Lists collections (paginated) or fetches one collection by id. The paginated list returns summary nested datasets and runs; get by id returns nested datasets and runs.
  • Use this to discover collections or inspect one collection in depth.

PUT /data-collections/{data_collection_id} (Update data collection)

  • Updates optional name, dataset_ids, custom_run_ids, org_id, and/or project_id.
  • Use this to rename a collection, attach datasets or custom runs, or adjust organization scope.

DELETE /data-collections/{data_collection_id} (Delete data collection)

  • Permanently removes a data collection.
  • Use this when a collection is no longer needed.

Organizations​

GET /organizations/me (Get my organization)

  • Returns the organization for a project-scoped API key, including org admins and members.
  • Returns null for a private key (including an org member's private key) and for solo accounts with no organization.

Projects​

GET /projects/me (Get my projects)

  • Lists projects in the key's organization that the caller can access (all projects for org admins; member projects only for members).
  • Returns an empty array for a private key or when the key has no associated organization.

Getting started​

  1. Use your organization’s Aegis web app to sign in and create an API key intended for integrations.
  2. Send the key on every request to the routes in this section, as described below.

Authentication​

Every route under /runs, /evaluate, /data-collections, /dataset-types, /datasets, /organizations, and /projects requires:

Authorization: Bearer <token>

API key scoping​

API keys may be private (no project assignment) or project-scoped (assigned to one organization project). Scoping is always explicit when you create the key in the web app — omitting a project yields a private key (there is no implicit default-project fallback).

If you belong to an organization, you may use either kind of key:

  • A project-scoped key has organization visibility locked to that one project, and usage bills the organization pool.
  • A private key has personal visibility only (your own resources — not org-shared ones), but usage still bills the organization pool. Org-admin privileges do not apply through a private key. GET /organizations/me returns null and GET /projects/me returns [] for a private key. Filtering lists with shared=true is unavailable (404), because the key is not treated as organization-scoped for authorization.

A key whose stored project no longer matches your current organization (stale / cross-org) is rejected with 403 ("This API key is no longer valid. Create a new key.").

A project-scoped key is locked to its project:

  • List endpoints (GET /datasets, GET /data-collections, and their partial/id variants) return only resources assigned to that project — not resources visible only because they belong to a collection assigned to the project.
  • Get by id, update, and delete on a resource that is unassigned or assigned to a different project are treated as not found — 404, even when the resource is otherwise visible to your organization. Download run follows the same 404 rule. Download dataset returns 403 when the dataset exists but your key cannot access it (including wrong or missing project assignment). Reserve 403 ("This resource is assigned to a different project than this API key.") for create/attach/claim when an existing resource already belongs to another project, or when an explicit project_id in the request body conflicts with the key's project.
  • Create endpoints accept an optional project_id that must match the key's project when provided; omit it to use the key's project. A conflicting project_id returns 403.
  • Creating a standalone resource (no data collection) with a project-scoped key org-shares it and assigns it to the key's project automatically.
  • Claim/attach flows (linking to a data collection, attaching datasets/runs) can still target unassigned resources and force-assign them to the key's project; a resource already assigned to a different project returns 403.
  • A project-scoped key cannot target personal (non-org-shared) resources — 403 ("This API key is scoped to an organization project and cannot be used for personal resources.").

Solo accounts (no organization) use private keys without project restrictions; usage bills the personal balance.

Billing and balance​

Operations that enqueue real model work generally check your account balance before proceeding. That includes starting a dataset run, a custom run, and calling POST /evaluate. If your balance is insufficient, the server responds with 402 Payment Required (see HTTP status codes and error responses). Costs are tracked in USD — each evaluation records an evaluation_cost (the dollar amount charged for that evaluation).

For organization members, both project-scoped and private API keys bill the organization pool (not the personal balance). Solo accounts bill the personal balance.

HTTP status codes and error responses​

Responses use this shape: JSON with a detail field.

  • 200 — Success for GET/PUT/DELETE where a body is returned.
  • 201 — Resource created (POST runs, POST evaluate, POST data collections, POST datasets).
  • 204 — Success with no body (DELETE run, DELETE data collection, DELETE dataset).
  • 400 — Bad request: invalid payload, inactive metrics, missing, or inconsistent IDs, duplicate alias/name, conflicting state (for example the resource is still referenced elsewhere), etc.
  • 401 — Missing/invalid Authorization header, or API key not accepted.
  • 402 — Insufficient balance — returned before POST /runs/dataset, POST /runs/custom, or POST /evaluate if your balance is too low.
  • 403 — Forbidden: not allowed to perform the operation (ownership/org role), or API key scoping violation (stale/cross-org key, explicit project_id mismatch on create/update, claim/attach conflict with an already-assigned resource, scoped key on a personal resource).
  • 404 — Resource not found (run, metric, model, dataset, dataset type, data collection, no rows/CSV data, etc.), including when a project-scoped key cannot access the resource because it is unassigned or assigned to a different project (get/update/delete, and download run). Download dataset uses 403 when the dataset exists but is not accessible — see API key scoping.
  • 422 — Validation failed - invalid JSON body or field types on a POST/PUT (e.g. POST /runs/dataset, POST /runs/custom), or out-of-range query params (e.g. page / page_size).
  • 500 — Unexpected server error.