Skip to main content
Model purposes now use role. Replace these names together: models, requires.models, ctx.models, and t.usage.cacheRead.rate(role?) keep their behavior. Apply the same field replacements to consumers of show --json, Query and View data. Old API names have no forwarding aliases.

​
Replace removed fields

Removed fields fail at definition loading, including fields explicitly set to undefined. Codex and Claude Code native plugins remain Adapter configuration. Config reporters, Eval diff and metadata, both build/preparation concurrency controls, and Agent/Adapter implementations remain supported. Existing results stay readable. Stored usage and config identity encodings are unchanged, so experiments that do not use removed fields can continue carrying their existing results.

​
Experiment uses adapter

Replace defineExperiment({ agent: target }) with defineExperiment({ adapter: target }). Values returned by defineAgent and defineSandboxAgent work directly as adapter; no wrapper is required. Definitions containing agent fail, even when its value is explicitly undefined. The single-model model / reasoningEffort shorthand remains and normalizes to the default role; use models for multiple purposes. Do not combine the two forms. Renaming agent to adapter preserves config identity, so existing results can still carry.

​
t.note

Use t.note(label, value) to save data for review without adding an Assertion or affecting Verdict, score, or coverage. Values can be JSON or a Quantity with a unit; null is displayed as unavailable.
Labels must be unique within an Attempt. The limits are 64 notes and 64 KiB of JSON per note; violations throw TypeError. show @<locator> and View display Notes after Assertions. JSON and Query Attempt results include notes. show --exp <experiment-id> groups numeric notes by label and unit, displaying count, min, median, p90, and max. Historical results without notes return an empty list.

​
数值单位

Wrap numbers with ms, seconds, tokens, usd, ratio, count, or bytes from niceeval/expect when passing them into NiceEval. Each constructor returns a frozen Quantity with .value and .unit; perform arithmetic before wrapping. Non-finite inputs throw RangeError, and ratio also rejects values outside [0, 1].
measured and quantile preserve units. Supply matching units to atLeast, atMost, greaterThan, lessThan, and inclusive between; mixed units or unwrapped thresholds are type errors. If types are bypassed, incompatible comparisons are unavailable. cacheRead.rate() now requires a ratio(...) threshold. Unitless materials still use plain number thresholds; old Records without a unit remain unitless. Show and View format units as 512 ms, 61.2%, and $0.031.

​
Planned attempt

A Run’s planned execution position is now called a Planned attempt. An Attempt remains a published execution fact. Use niceeval.query/v2 for queries; the /inspection decoder no longer accepts v1. Replace slotId, expectedSlots, and originSlotId with plannedAttemptId, plannedAttempts, and originPlannedAttemptId. The Run and current-result slots collections are now plannedAttempts. Show reports Planned / Recorded / Missing attempts. Existing Records need no rewrite: persistent fields are mapped at the query boundary. For Adapter state, replace createSessionSlot / SessionSlot with createSessionKey / SessionKey. Concurrency waits now display waiting for capacity.

​
命名收敛

Retired names have no aliases. Definition and CLI parsing errors point to their replacements. Preparation caches are now called before caches, with BeforeCacheKey identifying them. Timing and diagnostic lifecycle names use *.before and *.after. Cache directories, cache keys, and Record encodings stay unchanged; readers project the new names, so existing caches and results need no rewrite. Docker profiles retain only decodeDockerExecutionProfile, defineDockerExecutionProfile, indexDockerProfiles, resolveDockerProfile, and DOCKER_EXECUTION_PROFILE_SCHEMA_VERSION. Replace their parser, constructor, and registry aliases. Internal identity helpers, Compose compilation, build scheduling, capture normalization, and profile policy constants are no longer exported from /sandbox. /loaders no longer exports captureLoadedFiles; /inspection no longer exports canonical JSON helpers. These internal functions have no replacement author API.

​
Plugin

definePlugin({ name, identity, ... }) returns an instance; definePlugin((options) => ({ name, identity, ... })) returns a single-argument factory. Compose conditions with plugins: [condition] and array spread. Use key to distinguish named instances and declare the behavior version in identity. Lifecycle fields are experiment, group, attempt, and sandbox, each accepting { before, after }; agent supplies built-in Agent extensions. See Plugin.

​
生命周期 before / after

Experiments, defineAgent, and defineSandboxAgent use before / after. Built-in Agent factories use the same fields. For manual cleanup, use exp <selector> --after; the host API uses experimentHost.after(), hasBefore, hasAfter, and after-required. Installation reporting uses AgentInstallationManifest, reportInstallation, and agentInstallation; the attachment filename is unchanged.