niceeval.config.ts

niceeval.config.ts holds project-wide defaults. Run identity belongs to experiments/ files, not to the root config.

import { defineConfig } from "niceeval";

export default defineConfig({
  judge: { model: "gpt-5.4-mini" },
  sandbox: "auto",
  maxConcurrency: 4,
  timeoutMs: 300_000,
});

judge

JudgeConfig

Default model and endpoint for t.judge.autoevals.*.

reporters

Reporter[]

Optional custom reporters. Use --junit <path> for a one-off JUnit report from the CLI.

maxConcurrency

number

Maximum attempts to run concurrently.

timeoutMs

number

Per-attempt timeout in milliseconds.

sandbox

SandboxOption

Default sandbox backend. Files are not uploaded automatically; sandbox evals call t.sandbox.uploadDirectory(...), uploadFiles(...), or writeFiles(...) in test(t).