> ## Documentation Index
> Fetch the complete documentation index at: https://niceeval.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Eval an AI Agent Application with a Hand-Written Adapter

> A runnable AI SDK v6 web agent eval project covering a hand-written adapter, tool calls, image understanding, multi-turn sessions, model comparison, and dual observability.

This project shows how to use `defineAgent` to connect an existing HTTP agent to NiceEval. It's a good fit for applications that need control over their own protocol and event mapping.

* [View the full source](https://github.com/CorrectRoadH/niceeval/tree/main/examples/zh/ai-sdk)
* Local directory: `examples/zh/ai-sdk/`

<Note>
  If your application uses AI SDK v7's UI Message Stream, prefer the built-in `uiMessageStreamAgent` instead. See the corresponding non-intrusive example: [AI SDK v7](/docs/examples/integrations/ai-sdk-v7).
</Note>

## What this project demonstrates

| Part              | Implementation                                                                           |
| ----------------- | ---------------------------------------------------------------------------------------- |
| System under test | AI SDK v6 web agent, HTTP `POST /api/turn`                                               |
| Adapter           | `defineAgent` + custom `AgentEvent` → `StreamEvent` mapping                              |
| Evals             | Weather tool, image understanding, multi-turn text, multi-turn image context             |
| Experiment        | Each file pins one model; the group compares DeepSeek against GPT                        |
| Observability     | The app keeps reporting to Langfuse while also sending this turn's OTLP data to NiceEval |

The adapter is only responsible for connecting and translating. The URL and model belong to the experiment; evals only describe the interaction and what a good result looks like. This separation lets the same evals be reused across different models and deployment instances.

## Directory structure

```text theme={null}
examples/zh/ai-sdk/
├── src/                     # web agent under test
├── adapter/adapter.ts       # hand-written NiceEval adapter
├── evals/                   # tool, image, and multi-turn evals
├── experiments/             # model comparison
└── niceeval.config.ts
```

## Run it

This example calls real models — there's no mock mode. Copy the environment variables first and fill in credentials for an OpenAI-compatible service:

```bash theme={null}
cd examples/zh/ai-sdk
pnpm install
cp .env.example .env
pnpm dev
```

In a second terminal, run:

```bash theme={null}
cd examples/zh/ai-sdk
pnpm exec niceeval list
pnpm exec niceeval exp compare-models weather-tool
pnpm exec niceeval view
```

## Next steps

* [Connect Your Agent](/docs/tutorials/connect-your-agent): Start from a minimal adapter and connect your own protocol.
* [Write Send](/docs/tutorials/write-send): Gradually fill in sessions, tools, HITL, and OTel.
* [Experiment Matrix](/docs/tutorials/experiments): Organize model and configuration comparisons.