Skip to main content
When a repository has several Experiments and many Evals, confirm their definitions and preview the plan first. Do not guess an ID from a filename or start a batch that may select the wrong work.

​
List runnable Experiments

Each line shows the Experiment ID, description, Agent, model, Attempt count, selected Eval count, and labels. exp list only discovers and selects; it creates no Invocation, Sandbox, or model call. Use JSON when a script needs the complete list:

​
Narrow the scope with positional arguments

The first positional argument selects an Experiment ID, directory, or filename prefix. Later arguments narrow only the Evals already selected by those Experiments, using their ID prefixes:
An exact Experiment ID takes precedence over a prefix. A directory segment must match exactly. An Eval prefix cannot add a task back that is outside the Experiment’s long-lived scope. Use --tag when you only want Evals with a particular tag:
Put the long-lived scope in an Experiment’s evals. Positional arguments and --tag only narrow one Invocation temporarily.

​
Check the plan with --dry

Add --dry before a real run:
The plan lists Evals, runtime configuration, Attempt ordinals, and whether each position will execute or adopt a historical result. It creates no Invocation, Run, Sandbox, or model call, and writes no JUnit file. Combine it with --json when a machine needs to read the plan:
After you confirm the count, model, Attempts, and treatment of historical results, remove --dry and run the same command.

​
Handle selection failures

  • When no Experiment matches, the CLI lists browsable directories and suggests running exp <path> --dry first.
  • When no Eval matches, the CLI explicitly reports No evals selected; it does not fall back to the whole Experiment scope.
  • To inspect lifecycle commands rather than the execution matrix, use Inspect the Lifecycle Plan Before Running.
  • To decide which old results can be carried, continue with Rerun and Carry Results.