Skip to main content

Create dataset

Endpoint: POST /datasets

Description Creates a custom dataset from a CSV file. The server parses the file and stores records. Default selected metrics (default_selected_metrics: metric shortname → { threshold?, metric_args? }) are optional: omit them to store none. A later dataset run may send its own selected_metrics, or omit that field to reuse this stored default. You can also set name, description, and data_collection_id — the collection id must be one you can access (same rules as listing collections: yours or shared with your organization).

Sharing rules

The new dataset is always owned by you (your account is the individual owner). Sharing isn't accepted directly on this endpoint — visibility is derived from the data collection (if any), from an optional project_id, or from a project-scoped API key. Datasets created here are never org-only (org-only resources have no individual owner and can only be produced through admin flows).

  • Without data_collection_id:
    • Private (org_id = null, no project) when you use a private API key and omit project_id.
    • Org-shared and assigned to a project when you use a project-scoped API key (the key's project is applied and the dataset is org-shared automatically). A private key cannot org-share via project_id alone. See Introduction — API key scoping.
  • With data_collection_id:
    • The collection is loaded via claim semantics. When claimable, the dataset is created shared or private according to the collection's org_id and inherits the collection's project (not a separate project assignment). You must still be a member of that organization when the collection is org-shared. Unassigned collections can be claimed into the key's project; a collection already assigned to a different project returns 403.
    • The linked collection is private → the dataset is created private (no project until the collection or dataset is shared later).

Parameters

  • Body — multipart/form-data:
{
  "file": "binary (.csv file)",
  "default_selected_metrics": "string | null (JSON: see shape below)",
  "column_mappings": "string | null (JSON: Aegis field → CSV column name)",
  "name": "string | null",
  "description": "string | null",
  "data_collection_id": "integer | null",
  "project_id": "integer | null"
}
  • description is optional. Max 255 characters. Surrounding whitespace is trimmed; blank or whitespace-only values are stored as null. A longer value returns 422.
  • name is the stored dataset label (for example My evaluation set). The .csv check uses the uploaded file's own filename, case-insensitively. Any other extension returns 400 (Only CSV files are supported. Please upload a file with .csv extension.). A missing filename is the same 400.

project_id

  • Optional. On a private key, sending project_id alone does not org-share the dataset (private keys have personal visibility only). Use a project-scoped key to create org-shared datasets.
  • On a project-scoped key, must match the key's project when provided; omit to use the key's project (standalone creates are org-shared automatically).
  • Cannot be set on a dataset that will live inside a data collection — assign the collection instead.

default_selected_metrics is an optional JSON-encoded string. Omit it, pass null, or pass {} to store no metrics. When present and non-empty, it is a map keyed by metric shortname ("ans_corr"), where each value is an object:

{
  "<metric_shortname>": {
    "threshold": "integer (0–100) | null",
    "metric_args": "object | null"
  }
}
  • threshold is optional. When present it must be an integer between 0 and 100. Omit it (or send null) to use that metric's catalog default threshold at evaluation time.
  • metric_args is optional (null or omitted means "use the metric's defaults"). Keys are argument names declared by the metric and values should match the argument's declared type. Only the shape is checked here (metric_args must be an object): an unknown argument name (for example a typo like min_cout) is stored but ignored when the run evaluates, and a missing required argument or wrong-typed value surfaces later as an evaluation with is_success: false and the reason in explanation.
  • Keys are metric shortnames. Metric ids return 400. Unknown shortnames return 400. GET returns the same shortname keys.
  • A later POST /runs/dataset may omit selected_metrics to reuse this default_selected_metrics map, or send a different map for that run only (the dataset default is unchanged).

column_mappings is an optional JSON-encoded string. Decoded, it maps each Aegis field to the matching CSV column header:

{
  "prompt": "question",
  "input": "user_input",
  "context": "background",
  "output": "model_answer",
  "golden_answer": "expected_answer",
  "external_id": "upstream_row_id"
}
  • Keys must be Aegis field names: external_id, prompt, input, context, output, golden_answer. Include only the fields you want to import.
  • Values are the exact CSV column names (case-insensitive match against the file header).
  • Each CSV column can be mapped to only one field. Comparison is case-insensitive, so question and Question are the same column. Two fields that share a column return 400 (Each CSV column can be mapped to only one field. 'question' is mapped to input, output.).
  • Omit column_mappings (or pass null) to auto-detect columns whose headers already match Aegis field names (including external_id).
  • The CSV must still include at least one of prompt, input, context, output, or golden_answer. An external_id column alone is not enough to create a dataset.

Error responses

  • 401 — Authentication failed.
  • 403 — API key scoping violation (project_id conflicts with a project-scoped key, collection/dataset project mismatch, scoped key on a personal resource).
  • 422 — Missing required form parts or invalid multipart fields.
  • 400 — File name does not end in .csv, invalid CSV, empty file, bad encoding, invalid default_selected_metrics / column_mappings (including Invalid default_selected_metrics format. Must be valid JSON.; default_selected_metrics must be a JSON object mapping metric shortnames to {threshold, metric_args} objects.; two fields mapped to the same CSV column — Each CSV column can be mapped to only one field. '<column>' is mapped to <fields>.); metric id used as a map key (Invalid metric key '<id>'. Use a metric shortname, not a metric id.); unknown or inactive metric shortnames (The following metrics do not exist: …, The following metrics are not active: …); invalid thresholds or metric_args; duplicate metric keys; agentic metrics that score a trace or span (These metrics score a trace or a span and cannot be used on a dataset or custom run: <shortnames>.); duplicate dataset name for your account or organization. This endpoint always creates a custom dataset, so proprietary-dataset errors do not apply.
  • 404 — Dataset type not found; or data collection not found or not claimable for data_collection_id.
  • 500 — Server error.

Responses

  • 201 — dataset object with the same JSON shape as Get dataset by id. The example is a private dataset you own, created with a private API key, so can_share is false. On create, runs and evaluations are usually empty arrays until you start a dataset run; nested user_id and api_key_id fields are typically populated when present, but may be null per that shape.

Example response (201)

{
  "id": 42,
  "user": { "id": 7, "email": "analyst@acme.com" },
  "dataset_type_id": 2,
  "author_email": "analyst@acme.com",
  "name": "My evaluation set",
  "description": "Evaluation prompts for support QA",
  "default_selected_metrics": {
    "ans_corr": { "threshold": 70, "metric_args": null },
    "faith": { "threshold": 80, "metric_args": { "ignore_extra_keys": true } }
  },
  "structure": ["external_id", "prompt", "output", "golden_answer"],
  "column_mappings": {
    "external_id": "row_id",
    "prompt": "question",
    "output": "model_answer",
    "golden_answer": "expected_answer"
  },
  "data_collection_id": null,
  "org_id": null,
  "project_id": null,
  "permissions": {
    "can_edit": true,
    "can_delete": true,
    "can_share": false,
    "can_assign_project": false,
    "can_move_collection": true
  },
  "created_at": "2026-04-02T12:00:00Z",
  "updated_at": "2026-04-02T12:00:00Z",
  "extra_columns": [],
  "dataset_type": {
    "id": 2,
    "name": "CUSTOM",
    "label": "Custom",
    "description": null
  },
  "records": [
    {
      "id": 1001,
      "user_id": 7,
      "dataset_id": 42,
      "external_id": "row-42",
      "prompt": "What is the refund policy?",
      "input": null,
      "context": null,
      "output": "Refunds are available within 30 days.",
      "golden_answer": "30-day refund window.",
      "extra_values": {},
      "created_at": "2026-04-02T12:00:00Z",
      "updated_at": "2026-04-02T12:00:00Z"
    }
  ],
  "runs": [],
  "evaluations": [],
  "last_run_id": null
}

curl

curl -X POST "https://api.aegisevals.ai/api/v1/datasets" \
  -H "Authorization: Bearer sk_00000000000000000000000000000000" \
  -F "file=@/path/to/data.csv" \
  -F 'default_selected_metrics={"ans_corr":{"threshold":70,"metric_args":null},"faith":{"threshold":80,"metric_args":{"ignore_extra_keys":true}}}' \
  -F 'column_mappings={"external_id":"row_id","prompt":"question","output":"model_answer","golden_answer":"expected_answer"}' \
  -F 'name=My evaluation set' \
  -F 'description=Evaluation prompts for support QA'