Skip to main content
defineConfig is the default export from niceeval.config.ts at the project root and holds only project-level defaults. Agent, model, Flags, Attempts, and Experiment budgets belong in defineExperiment files under experiments/.

​
Config fields

​
name

The project name, shown in the hero (<h1>) at the top of niceeval view; it falls back to a generic title when omitted. It can be a string, or text in multiple locales (such as { en: "...", "zh-CN": "..." }) that switches with the view language.

​
workspace

The workspace root uploaded into a Sandbox. It defaults to the project root, and an Eval’s Sandbox view starts from here.

​
judge

The project-level default Judge configuration (model / baseUrl / apiKeyEnv). EvalDef.judge can override it per Eval.

​
reporters

The project-level default Reporter list, such as persisting or uploading results. EvalDef.reporters merges with it.

​
maxConcurrency

The project-level default concurrency limit. The CLI Flag or Experiment setting with the same name has higher priority; there is no environment-variable layer.

​
maxBuildConcurrency

Concurrency for preparing Sandbox images at the Run level. It is independent of Attempt concurrency and defaults safely to 2 when omitted.

​
timeoutMs

The project-level default timeout for one Attempt, in milliseconds. The CLI Flag, Experiment, or EvalDef setting with the same name takes priority.

​
telemetry

OTLP receiver configuration: the only entry point inside a niceeval project. It does not read NICEEVAL_OTLP_* environment variables. port pins the receiving port. In fixed-port mode, a long-running service points OTEL_EXPORTER_OTLP_ENDPOINT once at http://localhost:<port>/v1/traces, and does not need to change it for any number of Eval runs. When omitted, each run dynamically assigns a temporary port and supplies it to the Adapter through ctx.telemetry. The tradeoff is that only one niceeval process on the machine can use a fixed port at a time, and an error occurs if another process already uses that port. Choose an available port and write it here. host is the receiver hostname reported to the Adapter, not the listening address, which is always 0.0.0.0. It defaults to "127.0.0.1". Override it only after providing a controlled tunnel or reachable route. A Docker Sandbox puts the receiver inside the Sandbox by default and does not rely on an implicit host gateway.

​
pricing

User overrides or additions to the built-in price table (o11y/prices.json), looked up by model. See Observability · Usage and Cost. A key can be an exact model name or a provider/* wildcard, which applies a self-hosted or gateway discount in bulk by Provider. Exact keys take priority over wildcards. pricing drives only the estimatedCostUSD estimate from estimateCost; it is unrelated to usage.costUSD, the observed gateway cost. Both facts exist independently and neither fills in for the other. This is a runtime/Config price table; fixed Inspection operations do not take it as another input.
Cost boundary: this pricing provides only Runner Config/runtime price-table input. The Runner produces estimatedCostUSD from the model, reported token usage, and this table whether or not Usage.costUSD exists. The Experiment budget and t.maxCost() use that estimate; Usage.costUSD does not. Fixed Inspection operations do not turn runtime pricing configuration into another input.
NiceEval keeps environment preparation in ordinary code. An Eval writes its own required files in test(t), and an Agent’s preparation belongs in Adapter setup. Config deliberately has no sandbox field and does not auto-detect a Provider. Declare a template-bearing SandboxLayer on the Eval or Experiment, so link planning can determine its owner and pairing result before it creates any resources.