Documentation Index
Fetch the complete documentation index at: /docs/llms.txt
Use this file to discover all available pages before exploring further.
值断言、作用域断言、LLM-as-judge、test-as-scoring 和效率检查。
| 目标 | 推荐机制 |
|---|---|
| 回复包含固定子串或命中正则 | includes |
| 结构化 JSON 完全匹配 | equals |
| 工具调用是否发生 | t.calledTool / t.notCalledTool |
| 输出是否语义正确 | t.judge.* |
| 代码是否工作 | t.check(await t.sandbox.runCommand("npm", ["test"]), commandSucceeded()) |
| 成本是否可控 | t.maxTokens / t.maxCost |
import { includes, equals } from "niceeval/expect";
t.check(t.reply, includes("refund"));
t.check(turn.data, equals({ intent: "refund" }));
t.check(t.reply, includes(/order #\d+/));
includes 同时接受子串和正则；matches 只接受 Standard Schema / Zod schema，用于结构校验，不做正则匹配。值断言适合精确、稳定、低歧义的结果。
t.succeeded();
t.calledTool("search");
t.usedNoTools();
t.sandbox.fileChanged("src/app.ts");
t.check(await t.sandbox.runCommand("npm", ["test"]), commandSucceeded());
t.judge.autoevals.factuality("Refunds are available within 30 days.", { on: t.reply }).atLeast(0.8);
t.judge.autoevals.closedQA("The answer is specific, polite, and does not invent policy.", { on: t.reply }).atLeast(0.75);
t.check(t.reply, includes("refund").gate());
t.check(t.reply, includes("friendly tone").atLeast(0.7));
gate：失败就是失败。soft：保留分数，用于比较质量。.eval.ts 里运行项目测试或临时探针：
import { commandSucceeded, includes } from "niceeval/expect";
const testResult = await t.sandbox.runCommand("npm", ["test"]);
t.check(testResult, commandSucceeded());
const src = await t.sandbox.readSourceFiles({ extensions: ["ts"] });
t.check(src.text(), includes(/z\.object\s*\(/));
t.maxTokens(25_000);
t.maxCost(0.1);
此页面对您有帮助吗？