> ## Documentation Index
> Fetch the complete documentation index at: https://niceeval.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Set up evals for your agent project

> Install NiceEval, write three files, and run the first eval against your own application in about 10 minutes.

Connecting your application takes three files: an **Adapter** that knows how to call it, an **Experiment** that fixes the subject under test and the run count, and an **eval** that defines assertions. The steps below give the shortest complete path, covering both automatic setup with a coding agent and manual setup.

## Set up with a coding agent (recommended)

<Steps>
  <Step title="Set up">
    ```text theme={null}
    READ https://niceeval.com/INIT.md and set up niceeval for this repo: install it, integrate it with this project, and run the first eval end to end.

    ```
  </Step>

  <Step title="Run evals">
    ```bash theme={null}
    pnpm exec niceeval exp <experiment-name>
    ```
  </Step>

  <Step title="View results">
    ```bash theme={null}
    pnpm exec niceeval show           # all results still valid for the current project
    pnpm exec niceeval view           # the same Sample in the web viewer
    ```
  </Step>
</Steps>

## Manual setup

```bash theme={null}
pnpm add -D niceeval
pnpm exec niceeval init
```

### Adapter

The Adapter connects your Agent to [NiceEval](https://niceeval.com/). Three integration tiers, from non-intrusive connection to modifying your agent's configuration, unlock different eval capabilities — see [Tier](/docs/explanation/tier).

```ts theme={null}
// agents/my-agent.ts
import { completeEvidenceCoverage, defineAgent } from "niceeval/adapter";

export default defineAgent({
  name: "my-agent",
  evidenceCoverage: completeEvidenceCoverage,
  async send(input, ctx) {
    // Example endpoint: replace with your agent's real endpoint (HTTP, CLI, or SDK all work, as long as send() can get the reply text)
    const r = await fetch("http://localhost:3000/chat", {
      method: "POST",
      headers: { "content-type": "application/json" },
      body: JSON.stringify({ message: input.text, model: ctx.model }), // ← experiment.model flows through ctx.model here
      signal: ctx.signal,
    });
    const body = await r.json();
    return {
      status: r.ok ? "completed" : "failed",
      events: [{ type: "message", role: "assistant", text: body.reply }],
    };
  },
});
```

### Experiment

```ts theme={null}
// experiments/my-agent.ts
import { defineExperiment } from "niceeval";
import myAgent from "../agents/my-agent.ts";

export default defineExperiment({
  description: "my-agent baseline",
  model: "gpt-4o",
  agent: myAgent,
  attempts: 1,
});
```

### Eval

```ts theme={null}
// evals/refund-policy.eval.ts
import { defineEval } from "niceeval";
import { includes } from "niceeval/expect";

export default defineEval({
  description: "Refund policy question",
  async test(t) {
    const turn = await t.send("What is your refund policy?");
    await turn.succeeded().stopOnFailure();
    t.check(t.reply, includes("30 days"));
  },
});
```

```bash theme={null}
pnpm exec niceeval exp my-agent   # run it
pnpm exec niceeval show --experiment my-agent  # read all still-valid results in the terminal
pnpm exec niceeval view --experiment my-agent  # browse the same Sample locally
```

At this point, the first eval has run successfully.

This minimal Adapter only supports **single-turn** calls; the second turn does not carry the first turn's history. Multi-turn sessions, tool-call events (used by `t.calledTool()`), HITL, and tracing are all optional capabilities you add later. Existing evals do not need to change when you extend the setup by following [Connect Your Agent](/docs/tutorials/connect-your-agent).

## Compare against a complete project

Once the first eval runs successfully, go to [Examples](/docs/examples) and pick a complete project by subject under test. Those pages ship runnable source code; general operating steps live in the matching Tutorials task page.

## Put it in CI

```yaml theme={null}
name: evals
on: [pull_request]
jobs:
  evals:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
      - run: npm ci
      - run: npm exec niceeval exp my-agent
```

<Tip>
  Read [Authoring Evals](/docs/tutorials/authoring) and [Evaluation kinds](/docs/tutorials/evaluation-kinds), then replace the example with your real scenario.
</Tip>