Let a Coding Agent Iterate from Results

A Coding Agent can complete a clear feedback loop from the command line: read the installed documentation, run an Experiment, read the final receipt, use its runIds to inspect results, update the program or evals, and run again.

Start with the installed documentation

Tell the Agent to start with the documentation installed in the project instead of guessing an API:

Read node_modules/niceeval/INDEX.md first.
Then read the pages relevant to the current task from that index.
Run Experiments with commands, and use the final receipt to select the Runs to inspect.

That keeps the Agent’s instructions aligned with the version installed in the project.

Run and retain the receipt

npx niceeval exp checkout --json | tee .niceeval-invocation.ndjson

Every --json line is feedback from the current process. Progress and diagnostics describe what is happening in this run. Exactly one final receipt contains:

{
  "invocationId": "01J8ZK3M6P4T7V9X2C5N8QW0RY",
  "runIds": ["01J9ZK3M6P4T7V9X2C5N8QW0RY"],
  "startedAt": "2026-08-09T10:00:00.000Z",
  "completedAt": "2026-08-09T10:01:00.000Z",
  "completion": "completed"
}

The Agent should treat the final receipt as the handoff for this Invocation. It is not a persisted results protocol. Read business facts from the quiescent Record through its runIds.

Inspect results with Run IDs

After the run ends, inspect the Run’s Report:

npx niceeval show --run 01J9ZK3M6P4T7V9X2C5N8QW0RY
npx niceeval view --run 01J9ZK3M6P4T7V9X2C5N8QW0RY --no-open

show and view select a Sample before they form one fixed ReportInput. The Agent should distinguish these states:

Visible state	Next step
`failed`	Inspect the task result and assertions, then correct the system under test or eval logic.
`errored`	Inspect diagnostics, the Adapter, Sandbox, or credentials.
`not-recorded`	Find why that expected slot has no Member.
`unavailable`	Check whether the channel was not collected or does not apply.
`unsupported`	Address only the viewing work that depends on the channel.
`invalid`	Repair the named file or reference before reading again.

Run again after an edit

Have the Agent test one verifiable hypothesis at a time: change the program or eval, run the same scope, then read the new receipt and Run. To confirm that every slot executes again:

npx niceeval exp checkout --rerun all --json

When an existing Attempt is adopted automatically, --dry explains why. Reasons for carrying and accepting are in the target Run’s niceeval.actions channel. See Rerun and Carry Results.

Boundaries for Record edits

A Record is an editable fact dataset. An Agent may edit business channels only after the directory is quiescent, then run show or view again. Do not let a Runner, active reader, or AnalysisSample/ReportInput-construction phase operate on the same directory as a manual edit. Later Report execution and site writing do not access the Record. To share a result with someone else, use Publish a Static Report, not feedback from the current process.

Get Started

Connect Your Agent

Write Evals

Run, Observe & Publish

Troubleshooting

Let a Coding Agent Iterate from Results

Start with the installed documentation

Run and retain the receipt

Inspect results with Run IDs

Run again after an edit

Boundaries for Record edits

​Start with the installed documentation

​Run and retain the receipt

​Inspect results with Run IDs

​Run again after an edit

​Boundaries for Record edits

Start with the installed documentation

Run and retain the receipt

Inspect results with Run IDs

Run again after an edit

Boundaries for Record edits