Skip to main content

Sentence Count Match (sent_ct_match)

Contents​

Metric description​

Sentence count match counts sentences in the output and optionally enforces minimum and maximum counts. Sentence boundaries follow the implementation’s tokenizer (minimum words per sentence can be configured).

How to interpret the score​

  • 100: the count is within configured bounds (or no bounds were set and counting succeeded).
  • 0: outside bounds or the text could not be evaluated as required.

API usage​

Prerequisites

After the environment variables are configured, the next step is to create a JSON payload for the custom-runs request. For a field-by-field description of the payload (top-level keys, evaluations, and each row in data), see Custom run request body.

Shortname: sent_ct_match

Default threshold: 100

Structural metrics run without an LLM (deterministic checks). Your run may still include model_slug where the API expects it; scoring does not depend on it for this category.

Inputs (each object in data)

  • output (str, required): Text to segment into sentences.

metric_args

  • min_count (number optional): Minimum number of sentences required.

  • max_count (number optional): Maximum number of sentences allowed.

  • min_words_in_sentence (number optional): Minimum words in a sentence for it to count. Default: 1.

Eval metadata​

Structural metrics do not populate eval_metadata; the field is omitted or ull on the result object.

Example

import json
import os

import requests
from dotenv import load_dotenv

load_dotenv(override=True)

_API_KEY = os.getenv("AEGIS_API_KEY")
_BASE_URL = os.getenv("AEGIS_API_BASE_URL")
_CUSTOM_RUN_URL = f"{_BASE_URL}/runs/custom"


def post_custom_run(payload: dict) -> requests.Response:
    """POST JSON payload to Aegis custom runs; returns the raw response."""
    headers = {
        "Content-Type": "application/json",
        "Authorization": f"Bearer {_API_KEY}",
    }
    return requests.post(
        _CUSTOM_RUN_URL,
        headers=headers,
        data=json.dumps(payload),
    )


if __name__ == "__main__":
    data = [
        {"output": "First sentence. Second sentence."}
    ]

    payload = {
        "threshold": 100,
        "model_slug": "o4-mini",
        "is_blocking": True,
        "data_collection_id": None,
        "evaluations": [
            {
                "metrics": [
                    {
                        "metric": "sent_ct_match",
                        "metric_args": {
                            "min_count": 1,
                            "max_count": 10,
                            "min_words_in_sentence": 1,
                        },
                    },
                ],
                "threshold": 100,
                "model_slug": "o4-mini",
                "data": data,
            }
        ],
    }

    response = post_custom_run(payload)
    response.raise_for_status()
    print(json.dumps(response.json(), indent=2))