niceeval/expect Match 参考

niceeval/expect 只提供纯 Match factory。Match 比较一个候选值，不记录结果，也不决定判定或分数。t.check(value, match) 在调用时读取 value 并直接登记 Assertion；返回的 handle 只配置同一条 entry。

import { and, excludes, includes, pattern } from "niceeval/expect";

使用方式

t.check(t.reply, includes("confirmed"));
t.check(t.reply, and(pattern(/order #\d+/i), excludes("cancelled")));
const config = await t.check(rawConfig, pattern(/runtime/)).orStop();

入口	作用
`t.check(value, match)`	严格两个参数，调用时登记 Assertion；失败不中止后续代码
`await handle.orStop()`	同一 handle 的 async barrier；mismatch 或 below 时停止当前 continuation
`handle.score(n)` / `t.score(n)`	在计分制评估中让该项贡献分数，或直接登记 contribution

Matchers

下面每个 matcher 的签名和说明从 niceeval/expect 的源码生成，和当前实现保持同步。

`and`

export function and(
  first: BooleanMatch<unknown, unknown, MatchDomain>,
  ...rest: readonly BooleanMatch<unknown, unknown, MatchDomain>[]
): BooleanMatch<unknown, unknown, MatchDomain> { ... }

`or`

export function or(
  first: BooleanMatch<unknown, unknown, MatchDomain>,
  ...rest: readonly BooleanMatch<unknown, unknown, MatchDomain>[]
): BooleanMatch<unknown, unknown, MatchDomain> { ... }

`not`

export function not<T>(match: BooleanMatch<T, T, "value">): BooleanMatch<T, T, "value"> { ... }

`includes`

export function includes(text: string, options?: TextMatchOptions): BooleanMatch<string, string> { ... }

`excludes`

export function excludes(text: string, options?: TextMatchOptions): BooleanMatch<string, string> { ... }

`pattern`

export function pattern(expression: RegExp, options?: TextMatchOptions): BooleanMatch<string, string> { ... }

`similarity`

export function similarity(expected: string): ScoreMatch<string> { ... }

归一化 Levenshtein similarity；它是连续分数，不携带默认阈值或 verdict 策略。

`includesUrl`

export function includesUrl(min = 1): BooleanMatch<string, string> { ... }

至少含 min 个去重 http(s) URL 的纯文本 matcher。

`hasSections`

export function hasSections(min = 2): BooleanMatch<string, string> { ... }

至少含 min 个 Markdown heading 的纯文本 matcher。

`isDefined`

export function isDefined<T = unknown>(label?: string): BooleanMatch<T, Exclude<T, null | undefined>> { ... }

`isTrue`

export function isTrue<T = unknown>(label?: string): BooleanMatch<T, T & true> { ... }

value === true 的 refinement matcher。

`isFalse`

export function isFalse<T = unknown>(label?: string): BooleanMatch<T, T & false> { ... }

value === false 的 refinement matcher。

`commandSucceeded`

export function commandSucceeded<T = unknown>(): BooleanMatch<T, T & { readonly exitCode: 0 }> { ... }

CommandResult-like candidate 的 exitCode === 0 matcher；不读取 stdout / stderr。

`equals`

export function equals<const T>(expected: T): BooleanMatch<unknown, T> { ... }

`matches`

export function matches<S extends StandardSchema>(
  schema: S,
): BooleanMatch<unknown, StandardSchema.InferInput<S>, "value"> { ... }

`satisfies`

export function satisfies<T, R extends T = T>(
  label: string,
  predicate: (value: T) => boolean | Promise<boolean>,
): BooleanMatch<T, R> | BooleanMatch<T, T> { ... }

`defineValueMatch`

export function defineValueMatch<T, R extends T = T>(spec: {
  readonly name: string;
  readonly evaluate: (value: T) => boolean | Promise<boolean>;
}): BooleanMatch<T, R> | BooleanMatch<T, T> { ... }

`defineScoreMatch`

export function defineScoreMatch<T>(spec: {
  readonly name: string;
  readonly score: (value: T) => number | Promise<number>;
}): ScoreMatch<T> { ... }

`eventMatch`

export function eventMatch<K extends keyof EventOptionsByType>(
  type: K,
  options?: EventOptionsByType[K],
): EventMatch<Extract<AssertionEvent, { readonly type: K }>> { ... }

匹配工具调用

ToolMatch 比较一条 LogicalToolOccurrence。calledTool 接收 ToolMatch，或只按名称选择的薄糖；第二参数只表达次数。notCalledTool 使用相同 selector 表达零匹配。

import {
  commandMatch,
  jsonMatch,
  referencesAnyPath,
  toolMatch,
} from "niceeval/expect";

turn.calledTool(
  toolMatch("get_weather", {
    input: jsonMatch({ city: "Taipei", unit: "celsius" }),
    status: "completed",
  }),
  { count: 1 },
).label("精确查询天气");

turn.calledTool(
  toolMatch("read_file", {
    input: referencesAnyPath([".env", "secrets/**"]),
  }),
).label("读取受限路径");

turn.notCalledTool(commandMatch("rm", { argsStart: ["-rf"] }));

普通 JSON 用 jsonMatch，路径用 referencesAnyPath，命令 token 用 commandMatch。输出不存在、材料不完整或 HITL 尚未完成时，Match 会成为 unavailable，不会伪装成普通不匹配。完整签名、材料状态与三值计数规则见 Scoped assertions。

常见用法示例

t.check(t.reply, includes("Paris"));
t.check(t.reply, pattern(/order #\d+/i));
t.check(turn.data, equals({ intent: "refund" }));
t.check(turn.data, matches(OrderSchema));
t.check(turn.data, satisfies("返回数组", Array.isArray));
t.check(t.reply, similarity("The answer explains the refund window")).atLeast(0.8);

组合与三态

t.check(t.reply, and(includes("required"), excludes("forbidden")));
t.check(t.reply, or(includes("approved"), includes("accepted")));

同一 and / or 里的 Match 必须属于同一 domain。布尔 Match 的内部结果是 matched、mismatched 或 unavailable：or 只有在没有子项命中且至少一项证据不完整时才返回 unavailable，不会把未知当成否定。Match 自身不持有判定、计分或控制流；.atLeast(n)、.score(n) 与 .orStop() 属于登记后的 AssertionHandle。下面列出 Match 值本身的稳定字段。品牌和 evaluator 是模块私有实现，不属于作者 API。

`Match`

`domain`

readonly domain: D;

`name`

readonly name: string;

`BooleanMatch`

`kind`

readonly kind: "boolean";

`[matchRefinementBrand]`

readonly [matchRefinementBrand]: () => R;

`ScoreMatch`

`kind`

readonly kind: "score";

`atLeast`

atLeast(threshold: number): ThresholdedScoreMatch<T>;

自定义 matcher

import { defineValueMatch } from "niceeval/expect";

const validEmail = () =>
  defineValueMatch<string>({
    name: "valid email",
    evaluate(value) {
      return value.includes("@");
    },
  });

t.check(t.reply, validEmail());

布尔自定义 Match 用 defineValueMatch，连续分数用 defineScoreMatch。两者只实现单候选比较；读取文件、扫描事件、决定 coverage 或修改判定仍由 Assertion 登记入口与 Runner 负责。

API

适配器

结果与报告

CLI

niceeval/expect Match 参考

使用方式

Matchers

`and`

`or`

`not`

`includes`

`excludes`

`pattern`

`similarity`

`includesUrl`

`hasSections`

`isDefined`

`isTrue`

`isFalse`

`commandSucceeded`

`equals`

`matches`

`satisfies`

`defineValueMatch`

`defineScoreMatch`

`eventMatch`

匹配工具调用

常见用法示例

组合与三态

`Match`

`domain`

`name`

`BooleanMatch`

`kind`

`[matchRefinementBrand]`

`ScoreMatch`

`kind`

`atLeast`

自定义 matcher

​使用方式

​Matchers

​and

​or

​not

​includes

​excludes

​pattern

​similarity

​includesUrl

​hasSections

​isDefined

​isTrue

​isFalse

​commandSucceeded

​equals

​matches

​satisfies

​defineValueMatch

​defineScoreMatch

​eventMatch

​匹配工具调用

​常见用法示例

​组合与三态

​Match

​domain

​name

​BooleanMatch

​kind

​[matchRefinementBrand]

​ScoreMatch

​kind

​atLeast

​自定义 matcher

使用方式

Matchers

`and`

`or`

`not`

`includes`

`excludes`

`pattern`

`similarity`

`includesUrl`

`hasSections`

`isDefined`

`isTrue`

`isFalse`

`commandSucceeded`

`equals`

`matches`

`satisfies`

`defineValueMatch`

`defineScoreMatch`

`eventMatch`

匹配工具调用

常见用法示例

组合与三态

`Match`

`domain`

`name`

`BooleanMatch`

`kind`

`[matchRefinementBrand]`

`ScoreMatch`

`kind`

`atLeast`

自定义 matcher