failed or errored. You need to find the cause: the application answered wrong, the Adapter mapping missed something, the environment did not start, or an assertion is too strict.
Start from a specific Run or Attempt. Use niceeval show in the terminal and niceeval view in the browser; scripts and CI use query to read JSON.
Find the problem Attempt from a Run
First look at the run as a whole. The Run ID is in theNEXT panel when exp ends:
@ in front of it. Copy the one with the problem:
errored, it shows the phase that failed and the full error text. Looking here first is usually enough.
Choose what to look at by question
When the summary is not enough, add one flag for the direction you suspect:--execution shows a condensed execution outline. To expand one item, copy the ID next to it:
Find out why an assertion got this score
When the problem comes down to one assertion, usequery to get structured details. It returns that assertion’s contributions, measurements, and the Judge’s rationale:
show to assertion fields. For a complete request example, see View run results.
Cannot open results
Published results cannot be edited by hand. When you need different results, run again to create a new Run.
niceeval run recover only ends a Run whose original process is proven to have ended, and keeps every published Attempt. For the full steps, see Recover after a forced kill.
Read historical Records
The format version here is an internal identifier of the results file and is unrelated to NiceEval’s npm package version. Do not try to upgrade by editing the version field in the database.show, query, run list, run show, and view can read historical results files in the 0.15, 0.16, and 0.17 formats directly. When reading, NiceEval upgrades them automatically in a private copy. The original file, Runs, Attempts, and references stay unchanged, and you do not need to run an eval first.
When reading with --record, the file must be a complete single file that has already been handed over, and no other process can modify it while it is read. The command refuses to read when the file still has unmerged WAL, a run is still writing to it, or NiceEval cannot confirm that the writing process has ended. Do not delete the files next to it to get around the check.
When you run write commands such as exp normally in a project, NiceEval first backs up and validates, then upgrades supported old formats automatically. Reading a project in the current format never runs an upgrade or recovery.
Results are incomplete or cannot be compared
Scripts or CI first runpnpm --silent exec niceeval query discover to find the right query, then use explain to check the request, and finally use run to execute it. The result shows which results are missing, what issues exist, which evidence was used, and how to fix it.
query only outputs JSON and is not meant for people to read directly. To explain a score item by item, use attempt.get and attempt.assertion.detail instead of looking only at the summary score.
When you need a browser interface, run pnpm exec niceeval view --run <run-id> --no-open. Do not parse the View page, and do not query the tables in SQLite directly.
To compare two Runs, use runs.compare. Its exact and paired modes refuse outright when evals or pairs cannot be aligned, instead of quietly shrinking the denominator.
Read batched errors
In a run, the first time a known kind of error appears, the terminal immediately shows the full cause, the responsible party, the source location, and the next step. When the run ends, a summary lists the total count, the affected objects, and the source locations. Migration guides are shown in full only once perguideId, not repeated for each object.
Errors are grouped only when their code, owner, fix target, and guideId are all identical. The summary keeps at most 32 groups, and each group shows at most 32 affected objects and 32 source locations. Counts beyond that are stated separately, and every original error is still kept on its own. Unknown errors without a stable source are shown separately and are never merged just because their message or stack looks similar.
Source locations come from known call sites or source maps, and show unknown when none is found. NiceEval does not scan the whole project to report errors, and does not require the application to enable extra AST scanning.
Old project state blocks a run
If the project data predates the formats that support automatic upgrade, or an error says old directories such aslocks/ or sessions/ block writes, follow these steps to keep the history. The 0.15, 0.16, and 0.17 formats mentioned above are upgraded automatically and do not need this process.
- Stop any NiceEval process still running in the original project. Do not decide whether a lock file can be deleted based on its modification time.
- Keep the original project and the complete
.niceevaldirectory. To view history, use the NiceEval version that wrote the data, in the original project. If you are not sure which version that is, keep everything as is and check first. - Copy the configuration, eval cases, experiments, and other project files to a new directory, without
.niceeval, and install the current version of NiceEval in the new directory. - Run the original experiment in the new directory, for example
pnpm exec niceeval exp <experiment> --rerun all. New results go into the new directory’s results file, and the original project’s history stays as is.
niceeval state migrate --all manages user-level service state and cannot convert a project’s results file.
View shows old results
Whenniceeval view opens, it reads the results published at that moment and stays on them. When the page says new results are available, confirm the refresh to see the latest content. If the refresh fails, the page keeps the last results it could read.
For the normal workflow, see View run results. For command details, see CLI.