niceeval/expect Match Reference

niceeval/expect provides only pure Match factories. A Match compares one candidate value; it does not record a result or decide a Verdict or score. t.check(value, match) reads value and registers an Assertion immediately. The returned handle only configures that same entry.

import { and, excludes, includes, pattern } from "niceeval/expect";

How to use it

t.check(t.reply, includes("confirmed"));
t.check(t.reply, and(pattern(/order #\d+/i), excludes("cancelled")));
const config = await t.check(rawConfig, pattern(/runtime/)).orStop();

Entry point	Purpose
`t.check(value, match)`	Takes exactly two arguments and registers an Assertion when called; a failure does not stop later code.
`await handle.orStop()`	An async barrier on the same handle; stops the current continuation on a mismatch or result below threshold.
`handle.score(n)` / `t.score(n)`	Lets this item contribute a score in a scored Eval, or registers a contribution directly.

Matchers

The signature and description for every matcher below are generated from the niceeval/expect source and stay in sync with the current implementation.

`and`

export function and(
  first: BooleanMatch<unknown, unknown, MatchDomain>,
  ...rest: readonly BooleanMatch<unknown, unknown, MatchDomain>[]
): BooleanMatch<unknown, unknown, MatchDomain> { ... }

`or`

export function or(
  first: BooleanMatch<unknown, unknown, MatchDomain>,
  ...rest: readonly BooleanMatch<unknown, unknown, MatchDomain>[]
): BooleanMatch<unknown, unknown, MatchDomain> { ... }

`not`

export function not<T>(match: BooleanMatch<T, T, "value">): BooleanMatch<T, T, "value"> { ... }

`includes`

export function includes(text: string, options?: TextMatchOptions): BooleanMatch<string, string> { ... }

`excludes`

export function excludes(text: string, options?: TextMatchOptions): BooleanMatch<string, string> { ... }

`pattern`

export function pattern(expression: RegExp, options?: TextMatchOptions): BooleanMatch<string, string> { ... }

`similarity`

export function similarity(expected: string): ScoreMatch<string> { ... }

Normalized Levenshtein similarity. It is a continuous score and carries no default threshold or Verdict policy.

`includesUrl`

export function includesUrl(min = 1): BooleanMatch<string, string> { ... }

A plain-text matcher that requires at least min distinct http(s) URLs.

`hasSections`

export function hasSections(min = 2): BooleanMatch<string, string> { ... }

A plain-text matcher that requires at least min Markdown headings.

`isDefined`

export function isDefined<T = unknown>(label?: string): BooleanMatch<T, Exclude<T, null | undefined>> { ... }

`isTrue`

export function isTrue<T = unknown>(label?: string): BooleanMatch<T, T & true> { ... }

A refinement matcher for value === true.

`isFalse`

export function isFalse<T = unknown>(label?: string): BooleanMatch<T, T & false> { ... }

A refinement matcher for value === false.

`commandSucceeded`

export function commandSucceeded<T = unknown>(): BooleanMatch<T, T & { readonly exitCode: 0 }> { ... }

A matcher for CommandResult-like candidates whose exitCode === 0. It does not read stdout or stderr.

`equals`

export function equals<const T>(expected: T): BooleanMatch<unknown, T> { ... }

`matches`

export function matches<S extends StandardSchema>(
  schema: S,
): BooleanMatch<unknown, StandardSchema.InferInput<S>, "value"> { ... }

`satisfies`

export function satisfies<T, R extends T = T>(
  label: string,
  predicate: (value: T) => boolean | Promise<boolean>,
): BooleanMatch<T, R> | BooleanMatch<T, T> { ... }

`defineValueMatch`

export function defineValueMatch<T, R extends T = T>(spec: {
  readonly name: string;
  readonly evaluate: (value: T) => boolean | Promise<boolean>;
}): BooleanMatch<T, R> | BooleanMatch<T, T> { ... }

`defineScoreMatch`

export function defineScoreMatch<T>(spec: {
  readonly name: string;
  readonly score: (value: T) => number | Promise<number>;
}): ScoreMatch<T> { ... }

`eventMatch`

export function eventMatch<K extends keyof EventOptionsByType>(
  type: K,
  options?: EventOptionsByType[K],
): EventMatch<Extract<AssertionEvent, { readonly type: K }>> { ... }

Match tool calls

ToolMatch compares one logical tool occurrence. It owns occurrence quantifiers such as .exactly(n) and .atLeast(n). Pass a name, an unquantified ToolMatch, or a positive quantified Match to calledTool. notCalledTool uses an unquantified selector to express zero matches.

import {
  commandMatch,
  jsonMatch,
  referencesAnyPath,
  toolMatch,
} from "niceeval/expect";

turn.calledTool(
  toolMatch("get_weather", {
    input: jsonMatch({ city: "Taipei", unit: "celsius" }),
    status: "completed",
  }).exactly(1),
).label("Query weather precisely");

turn.calledTool(
  toolMatch("read_file", {
    input: referencesAnyPath([".env", "secrets/**"]),
  }),
).label("Read a restricted path");

turn.notCalledTool(commandMatch("rm", { argsStart: ["-rf"] }));

Use jsonMatch for ordinary JSON, referencesAnyPath for paths, and commandMatch for command tokens. When output is absent, material is incomplete, or HITL has not finished, a Match becomes unavailable; it does not pretend to be an ordinary mismatch. For complete signatures, material states, and three-valued counting rules, see Scoped assertions.

Common usage examples

t.check(t.reply, includes("Paris"));
t.check(t.reply, pattern(/order #\d+/i));
t.check(turn.data, equals({ intent: "refund" }));
t.check(turn.data, matches(OrderSchema));
t.check(turn.data, satisfies("returns an array", Array.isArray));
t.check(
  t.reply,
  similarity("The answer explains the refund window").atLeast(0.8),
).gate();

Composition and three states

t.check(t.reply, and(includes("required"), excludes("forbidden")));
t.check(t.reply, or(includes("approved"), includes("accepted")));

Matches within the same and / or must belong to the same domain. The internal result of a Boolean Match is matched, mismatched, or unavailable. or returns unavailable only when no child matches and at least one child’s evidence is incomplete; it does not treat an unknown result as a negative one. A Match itself has no Verdict, score, or control flow. ScoreMatch.atLeast(n) forms a threshold before registration. .score(n), parameterless .gate(), and .orStop() belong to the registered AssertionHandle. The following lists stable fields on Match values themselves. Brands and evaluators are module-private implementation details, not author API.

`Match`

`domain`

readonly domain: D;

`name`

readonly name: string;

`BooleanMatch`

`kind`

readonly kind: "boolean";

`[matchRefinementBrand]`

readonly [matchRefinementBrand]: () => R;

`ScoreMatch`

`kind`

readonly kind: "score";

`atLeast`

atLeast(threshold: number): ThresholdedScoreMatch<T>;

Custom matchers

import { defineValueMatch } from "niceeval/expect";

const validEmail = () =>
  defineValueMatch<string>({
    name: "valid email",
    evaluate(value) {
      return value.includes("@");
    },
  });

t.check(t.reply, validEmail());

Use defineValueMatch for a custom Boolean Match and defineScoreMatch for a continuous score. Both compare only one candidate. Reading files, scanning events, deciding coverage, and changing a Verdict remain the responsibility of Assertion registration entry points and the Runner.

API

Adapters

CLI

niceeval/expect Match Reference

How to use it

Matchers

`and`

`or`

`not`

`includes`

`excludes`

`pattern`

`similarity`

`includesUrl`

`hasSections`

`isDefined`

`isTrue`

`isFalse`

`commandSucceeded`

`equals`

`matches`

`satisfies`

`defineValueMatch`

`defineScoreMatch`

`eventMatch`

Match tool calls

Common usage examples

Composition and three states

`Match`

`domain`

`name`

`BooleanMatch`

`kind`

`[matchRefinementBrand]`

`ScoreMatch`

`kind`

`atLeast`

Custom matchers

​How to use it

​Matchers

​and

​or

​not

​includes

​excludes

​pattern

​similarity

​includesUrl

​hasSections

​isDefined

​isTrue

​isFalse

​commandSucceeded

​equals

​matches

​satisfies

​defineValueMatch

​defineScoreMatch

​eventMatch

​Match tool calls

​Common usage examples

​Composition and three states

​Match

​domain

​name

​BooleanMatch

​kind

​[matchRefinementBrand]

​ScoreMatch

​kind

​atLeast

​Custom matchers

How to use it

Matchers

`and`

`or`

`not`

`includes`

`excludes`

`pattern`

`similarity`

`includesUrl`

`hasSections`

`isDefined`

`isTrue`

`isFalse`

`commandSucceeded`

`equals`

`matches`

`satisfies`

`defineValueMatch`

`defineScoreMatch`

`eventMatch`

Match tool calls

Common usage examples

Composition and three states

`Match`

`domain`

`name`

`BooleanMatch`

`kind`

`[matchRefinementBrand]`

`ScoreMatch`

`kind`

`atLeast`

Custom matchers