> ## Documentation Index
> Fetch the complete documentation index at: https://niceeval.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# niceeval/expect matchers and custom assertion reference

> Reference for niceeval/expect: includes, equals, matches, similarity, satisfies, plus the shape assertions includesUrl and hasSections. Chain .gate() or .atLeast(0.7), or build custom matchers with makeAssertion.

`niceeval/expect` provides a set of composable matchers, passed to `t.check()` or `t.require()`. A matcher returns an `Assertion` with a default severity: `gate` or `soft`.

```ts theme={null}
import { includes, equals, matches, similarity, satisfies } from "niceeval/expect";
```

## How matchers are used

```ts theme={null}
t.check(t.reply, includes("confirmed"));
t.check(turn.data, equals({ intent: "refund" }));
t.require(turn.status, equals("completed"));
```

| Method      | On failure                              | Best for                                      |
| ----------- | --------------------------------------- | --------------------------------------------- |
| `t.check`   | Records the result, execution continues | Most assertions                               |
| `t.require` | Throws immediately, aborting the test   | Preconditions where continuing makes no sense |

## Matchers

The signature and description of each matcher below is generated from the `niceeval/expect` source, and stays in sync with the current implementation.

#### `includes`

```ts theme={null}
export function includes(needle: string | RegExp, opts?: MatchOptions): ValueAssertion { ... }
```

1 if `String(value)` contains the substring / matches the regex, otherwise 0. A hard gate by default. With `opts.stripComments`, only real code is examined.

#### `excludes`

```ts theme={null}
export function excludes(needle: string | RegExp, opts?: MatchOptions): ValueAssertion { ... }
```

The inverse of `includes`: 1 if it does not contain the substring / does not match the regex, otherwise 0. A hard gate by default. With `opts.stripComments`, only real code is examined.

#### `equals`

```ts theme={null}
export function equals(expected: unknown): ValueAssertion { ... }
```

1 on deep equality, otherwise 0. A hard gate by default.

#### `matches`

```ts theme={null}
export function matches(schema: unknown): ValueAssertion { ... }
```

Validates `value` against a schema — this is not regex matching, it's Standard Schema / zod-style structural validation.
Prefers Standard Schema (`schema['~standard'].validate`), otherwise falls back to zod-style
`.safeParse` / `.parse`. 1 if validation passes, otherwise 0; any exception → 0. A hard gate by default.

#### `similarity`

```ts theme={null}
export function similarity(expected: string): ValueAssertion { ... }
```

Pure string edit distance, not semantic similarity — a normalized Levenshtein distance in \[0,1] (1 − edit distance / length of the longer string),
with no understanding of meaning, so paraphrasing / reordering scores low. A soft score by default, threshold 0.6.

#### `includesUrl`

```ts theme={null}
export function includesUrl(min = 1): ValueAssertion { ... }
```

1 if the text contains at least min (default 1) deduplicated http(s) links, otherwise 0. A hard gate by default.
A shape assertion for "does the answer cite real sources": when no Judge key is available, this is the lowest-cost
backstop for "cites at least one source link" — echoing the question back cannot pass it; fabricated links are left
for negative cases or the Judge to catch.

#### `hasSections`

```ts theme={null}
export function hasSections(min = 2): ValueAssertion { ... }
```

1 if the text contains at least min (default 2) Markdown headings (line-leading # through ######), otherwise 0. A hard gate by default.
A shape assertion for "is the answer a structured document" — suited to research reports, design docs, and other
produce-type answers; a wall of flowing text with no section headings does not pass.

#### `satisfies`

```ts theme={null}
export function satisfies(predicate: (v: unknown) => boolean, label?: string): ValueAssertion { ... }
```

1 if the predicate is true, otherwise 0. A hard gate by default; `label` is added to the name for easier identification in reports.

#### `isDefined`

```ts theme={null}
export function isDefined(label?: string): ValueAssertion { ... }
```

1 if `value` is not null / not undefined, otherwise 0. Saves the boilerplate of `x !== undefined` + `isTrue`. A hard gate by default.

#### `isTrue`

```ts theme={null}
export function isTrue(label?: string): ValueAssertion { ... }
```

1 if `value === true`, otherwise 0. A boolean assertion with a label (for checks like `fileExists`). A hard gate by default.

#### `commandSucceeded`

```ts theme={null}
export function commandSucceeded(): ValueAssertion { ... }
```

1 if `CommandResult.exitCode === 0`, otherwise 0. A hard gate by default.

#### `isFalse`

```ts theme={null}
export function isFalse(label?: string): ValueAssertion { ... }
```

1 if `value === false`, otherwise 0. A boolean assertion with a label. A hard gate by default.

#### `makeAssertion`

```ts theme={null}
export function makeAssertion(spec: {
  name: string;
  severity?: Severity;
  threshold?: number;
  score: (value: unknown) => number | Promise<number>;
}): ValueAssertion { ... }
```

Custom assertion factory: give it a name / severity / threshold / score directly, and one call returns a ready-to-use `ValueAssertion` —
unlike `gate()`/`atLeast()`, it does not need a second chained call to set the level. `severity` defaults to `gate` when omitted.

### Common usage examples

```ts theme={null}
t.check(t.reply, includes("Paris"));
t.check(t.reply, includes(/order #\d+/i));
t.check(turn.data, equals({ intent: "refund" }));
t.check(turn.data, matches(OrderSchema)); // Standard Schema / Zod structural validation; use includes for regex matching
t.check(t.reply, similarity("The answer explains the refund window").atLeast(0.8));
t.check(turn.data, satisfies((value) => Array.isArray(value)));
```

## `gate` and `soft`

```ts theme={null}
t.check(t.reply, includes("required").gate());
t.check(t.reply, includes("nice to have").atLeast(0.7));
```

Every `Assertion` (i.e. `ValueAssertion`) returned by a matcher has these members; `.gate()` turns it into a hard gate, `.atLeast()` turns it into a soft threshold:

#### `name`

```ts theme={null}
readonly name: string;
```

#### `severity`

```ts theme={null}
readonly severity: Severity;
```

#### `threshold`

```ts theme={null}
readonly threshold?: number;
```

#### `isOptional`

```ts theme={null}
readonly isOptional?: boolean;
```

Marker set by chaining `.optional()`: when this assertion can't be scored, it is only recorded as unavailable, without dragging the attempt into `errored`.

#### `expected`

```ts theme={null}
readonly expected?: string;
```

A bounded text description of the expected condition (e.g. `contains "Brooklyn"`), included in `AssertionResult.expected` on failure.

#### `score`

```ts theme={null}
score(value: unknown): number | Promise<number>;
```

#### `gate`

```ts theme={null}
gate(threshold?: number): ValueAssertion;
```

Turns it into a hard-gate assertion: when the threshold isn't met (omitting `threshold` judges by `score > 0`), the whole eval is marked failed. Returns a new instance, does not mutate the original.

#### `atLeast`

```ts theme={null}
atLeast(threshold: number): ValueAssertion;
```

Turns it into a soft-threshold assertion: when `threshold` isn't met, this assertion is recorded as failed, but by default it does not drag down the whole eval's verdict;
under `--strict`, a soft-threshold failure also makes the whole eval's verdict count as failed. Returns a new instance, does not mutate the original.

#### `optional`

```ts theme={null}
optional(): ValueAssertion;
```

Allows this assertion's evidence to be absent: when it can't be scored, only records `outcome: "unavailable"`, without affecting the verdict.
Orthogonal to `severity` (severity determines whether it affects the quality verdict; `optional` determines whether evidence is allowed to be absent). Returns a new instance, does not mutate the original.

## Custom matchers

```ts theme={null}
import { makeAssertion, type Assertion } from "niceeval/expect";

function validEmail(): Assertion {
  return makeAssertion({
    name: "valid email",
    score(value) {
      return typeof value === "string" && value.includes("@") ? 1 : 0;
    },
  });
}

t.check(t.reply, validEmail());
```

`makeAssertion` takes a single spec object (`name`, optional `severity`/`threshold`, and `score`) and returns an `Assertion` directly; by convention you wrap it in a function of the same name and call that, rather than calling `makeAssertion` itself as the factory at the call site.