> ## Documentation Index
> Fetch the complete documentation index at: https://niceeval.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Built-in agent capabilities reference

> Which capabilities NiceEval's built-in claude-code, codex, and bub adapters implement, which assertions they map to, and known limits.

The three built-in sandbox agents (`claude-code`, `codex`, `bub`) do not implement exactly the same capabilities — capabilities have no declaration layer and are proven entirely by construction and actual behavior, see the [capabilities reference](/docs/reference/capabilities). This page inventories what each built-in agent does per capability, plus two built-ins for connecting AI SDK apps: the non-invasive HTTP adapter `uiMessageStreamAgent` (with HITL) and the result converter `turnFromAiSdk`.

## Capability overview

| Capability                            | Corresponding assertions / API                                                                                                                                                                                                                                              |
| ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Send/receive messages (base contract) | `t.send()` (can be called multiple times), `t.reply`, `turn.outputEquals` / `outputMatches`, status-based `t.succeeded()`                                                                                                                                                   |
| Event stream completeness             | The whole scoped assertion set: `calledTool` / `toolOrder` / `usedNoTools` / `maxToolCalls` / `messageIncludes` / `noFailedActions` / `event` / `eventOrder`, etc.; **negative assertions (`notCalledTool` and friends) are only trustworthy with a complete event stream** |
| Session continuation                  | Cross-turn memory assertions, `t.newSession()` session isolation                                                                                                                                                                                                            |
| HITL (human in the loop)              | `t.parked()`, `t.requireInputRequest()`, `t.respond()` / `t.respondAll()`                                                                                                                                                                                                   |
| `tracing`                             | trace decoding, the call waterfall in `niceeval view`                                                                                                                                                                                                                       |

## What each built-in agent implements

| Agent         | Send/receive | Event stream | Session continuation                   | HITL | tracing                                 | Notes                                                                                  |
| ------------- | ------------ | ------------ | -------------------------------------- | ---- | --------------------------------------- | -------------------------------------------------------------------------------------- |
| `claude-code` | ✅            | ✅            | ✅ (`claude --resume <id>`)             | ❌    | ✅ (`http/protobuf` → OTLP, beta toggle) | Built-in parser emits `compaction` events automatically; `t.event("compaction")` works |
| `codex`       | ✅            | ✅            | ✅ (`codex exec resume <id>`)           | ❌    | ✅ (`http/json` → OTLP)                  | Built-in parser emits `compaction` events automatically                                |
| `bub`         | ✅            | ✅            | ✅ (`--session-id` + tape continuation) | ❌    | ✅ (`http/protobuf` → OTLP)              | Built-in parser emits `compaction` events automatically                                |

All three are constructed with `defineSandboxAgent` (`Agent.kind` is always `"sandbox"`), so `t.sandbox.fileChanged()` / diff assertions, file IO, and command execution work on all three agents regardless of this table.

<Warning>
  None of the three built-in sandbox agents currently support HITL: their `send` only returns `"completed"` / `"failed"`, never `"waiting"`, and never emits `input.requested` events. When you need `t.respond()` / `t.requireInputRequest()`: if the system under test is an AI SDK `useChat` backend, use the built-in `uiMessageStreamAgent` below (v7 tool approval maps natively to HITL); for anything else write your own adapter: implement the `waiting` status + `input.requested` event + resume handoff — [`examples/zh/tier1/pi-sdk`](https://github.com/CorrectRoadH/niceeval/tree/main/examples/zh/tier1/pi-sdk) / [`examples/zh/tier1/claude-sdk`](https://github.com/CorrectRoadH/niceeval/tree/main/examples/zh/tier1/claude-sdk) both have ready-made hand-written HITL references.
</Warning>

## Per-agent details

### claude-code

* Connection: spawns `claude --print --dangerously-skip-permissions` in the Sandbox and reads back the latest transcript from `~/.claude/projects/**/*.jsonl`.
* Session continuation: appends `--resume <id>` when `ctx.session.id` has a value; the session id parsed from the transcript is written back via `ctx.session.capture()`.
* Auth: `ANTHROPIC_API_KEY`, optional `ANTHROPIC_BASE_URL`; configuration options are listed under `ClaudeCodeConfig` below.
* `tracing` is configured via the claude CLI's native OTLP trace spans (beta), protocol `http/protobuf`: setting `CLAUDE_CODE_ENABLE_TELEMETRY` and `CLAUDE_CODE_ENHANCED_TELEMETRY_BETA` (a beta toggle, must be explicitly enabled) in env hands the endpoint to the CLI; trace decoding shows a waterfall at the interaction / llm\_request / tool level.

### codex

* Connection: runs `codex exec --json` in the Sandbox (`codex exec resume <id> --json` when continuing), using stdout JSONL as the transcript. The command carries `--dangerously-bypass-approvals-and-sandbox` and `--dangerously-bypass-hook-trust`: nobody in the Sandbox can answer codex's interactive approval prompts, so hooks installed by a plugin or `postSetup` take effect without needing interactive trust confirmation.
* Auth: `CODEX_API_KEY` (not `OPENAI_API_KEY`), optional `CODEX_BASE_URL` for OpenAI-compatible proxies; configuration options are listed under `CodexConfig` below.
* `tracing` is configured through the `[otel.trace_exporter.otlp-http]` section of `~/.codex/config.toml`, protocol `http/json`.

### bub

* Connection: runs `bub run` + `--session-id` in the Sandbox and reads the tape from `~/.bub/tapes/<hash>.jsonl` as the transcript.
* Auth: `BUB_API_KEY` + `BUB_API_BASE` (OpenAI-compatible proxy); configuration options are listed under `BubConfig` below.
* `tracing` is injected via environment variables (`OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`, etc.), protocol `http/protobuf`.
* Installed via `uv tool install` (not an npm package); the first install builds a checkpoint cache to speed up later Sandboxes.

## Configuration options for the three built-in sandbox agents

### `ClaudeCodeConfig`

#### `apiKey`

```ts theme={null}
apiKey?: string;
```

Anthropic API key. Falls back to the `ANTHROPIC_API_KEY` env var if omitted.

#### `baseUrl`

```ts theme={null}
baseUrl?: string;
```

Custom API base URL (proxy / internal endpoint). Falls back to the `ANTHROPIC_BASE_URL` env var if omitted;
if neither is set, uses the Anthropic official endpoint (the claude CLI's default behavior).

#### `maxTurns`

```ts theme={null}
maxTurns?: number;
```

Maximum number of tool-use turns to run (→ `--max-turns`).
Caps the eval's cost ceiling; if omitted, uses the CLI's native default (unlimited).

#### `mcpServers`

```ts theme={null}
mcpServers?: McpServer[];
```

Additional MCP servers (written into the user-level `~/.claude.json` on each Sandbox setup).
The stdio form writes `command` (optionally with `args` / `env`); the Streamable HTTP form writes `url` (optionally with `headers`,
carried through verbatim into the request headers), landing as a \{ "type": "http", "url": …, "headers": … } entry.

#### `skills`

```ts theme={null}
skills?: SkillSpec[];
```

Skills to install into the Sandbox (a local directory/file, or a repo with an optional pinned ref and an optional enabled subset).
They land in the project-level `.claude/skills/<name>/`, where the claude CLI discovers them natively.

#### `plugins`

```ts theme={null}
plugins?: ClaudeCodePluginSpec[];
```

Claude Code native plugins (connect the marketplace first, then install the named plugin from it).

#### `settingsFile`

```ts theme={null}
settingsFile?: string;
```

Path to a complete Claude Code `settings.json` (official format) in the local project — resolved relative to the project
root running niceeval (the directory containing `niceeval.config.ts`), not a path inside the Sandbox; only relative
paths within the project root are accepted — paths containing `..`, absolute paths, `~` paths, and symlinks that
resolve outside the project root all error during setup. The raw bytes are uploaded verbatim as the user-level
`~/.claude/settings.json`, which is otherwise empty in the Sandbox (no inheriting host config, no merging, no
re-serializing); the reserved keys `model` and `env` appearing in the file error during setup. The manifest records
only the project-relative path and the byte SHA-256, never the file body.

#### `postSetup`

```ts theme={null}
postSetup?: SandboxHook[];
```

User hooks that run, in array order, after install (reusing SandboxHook's narrow context): executed once writing
settings, wiring MCP, installing Skills / plugins, and writing the manifest are all done — suited for process actions
that can only run once install artifacts are in place, such as a plugin's own setup script. A thrown error counts as
an infrastructure error (attempt errored).
See "Running scripts after install" in docs/feature/adapters/library/coding-agent-extensions.md.

#### `preTeardown`

```ts theme={null}
preTeardown?: SandboxHook[];
```

The teardown hook paired with `postSetup`: runs in reverse of `postSetup`'s ordering, before the agent's own teardown
step (a LIFO mirror — `postSetup` runs after the agent installs, `preTeardown` runs before the agent tears down), and
fires if and only if `postSetup`'s point in time was reached. A thrown error counts as an infrastructure error, folded
into a `teardown-failed` diagnostic by the teardown phase.
See "Running scripts after install" in docs/feature/adapters/library/coding-agent-extensions.md.

### `CodexConfig`

#### `apiKey`

```ts theme={null}
apiKey?: string;
```

Proxy / OpenAI API key. Falls back to the `CODEX_API_KEY` env var if omitted.

#### `baseUrl`

```ts theme={null}
baseUrl?: string;
```

OpenAI-compatible proxy base URL (e.g. `https://s2a.example.com/v1`). Falls back to the `CODEX_BASE_URL` env var if omitted.

#### `mcpServers`

```ts theme={null}
mcpServers?: McpServer[];
```

Additional MCP servers (appended into `~/.codex/config.toml` on each Sandbox setup).
The stdio form (`command`/`args`/`env`) writes the `command` line under \[mcp\_servers.\<name>];
the Streamable HTTP form (`url`/`headers`) writes the `url` line, with `headers` going into the
\[mcp\_servers.\<name>.http\_headers] subtable.

#### `skills`

```ts theme={null}
skills?: SkillSpec[];
```

Skills to install into the Sandbox (a local directory/file, or a repo with an optional pinned ref and an optional enabled subset).
They land in `.agents/skills/<name>/`, together with a discovery instruction written into AGENTS.md — codex has no native Skill tool like Claude Code, so merely installing the files does not make it read them (see memory/codex-no-native-skill-tool.md).

#### `plugins`

```ts theme={null}
plugins?: CodexPluginSpec[];
```

Codex native plugins (connect the marketplace first, then install the named plugin from it).

#### `configFile`

```ts theme={null}
configFile?: string;
```

Path to a complete Codex `config.toml` (official TOML format) in the local project — resolved relative to the project
root running niceeval (the directory containing `niceeval.config.ts`), not a path inside the Sandbox; only relative
paths within the project root are accepted — paths containing `..`, absolute paths, `~` paths, and symlinks that
resolve outside the project root all error during setup. The raw bytes are merged verbatim into the user-level
`~/.codex/config.toml`, which is otherwise empty in the Sandbox (no inheriting host config, no parse-and-rewrite);
the reserved keys `model`, `model_provider`, `model_providers`, `model_reasoning_effort`, `mcp_servers`, and `otel`
appearing in the file error during setup. The manifest records only the project-relative path and the byte SHA-256,
never the file body.

#### `postSetup`

```ts theme={null}
postSetup?: SandboxHook[];
```

User hooks that run, in array order, after install (reusing SandboxHook's narrow context): executed once writing
the main config, wiring MCP, installing Skills / plugins, and writing the manifest are all done — suited for process
actions that can only run once install artifacts are in place, such as a plugin's own setup script. A thrown error
counts as an infrastructure error (attempt errored).
See "Running scripts after install" in docs/feature/adapters/library/coding-agent-extensions.md.

#### `preTeardown`

```ts theme={null}
preTeardown?: SandboxHook[];
```

The teardown hook paired with `postSetup`: runs in reverse of `postSetup`'s ordering, before the agent's own teardown
step (a LIFO mirror — `postSetup` runs after the agent installs, `preTeardown` runs before the agent tears down), and
fires if and only if `postSetup`'s point in time was reached. A thrown error counts as an infrastructure error, folded
into a `teardown-failed` diagnostic by the teardown phase.
See "Running scripts after install" in docs/feature/adapters/library/coding-agent-extensions.md.

### `BubConfig`

#### `apiKey`

```ts theme={null}
apiKey?: string;
```

API key for the OpenAI-compatible proxy. Falls back to the `BUB_API_KEY` env var if omitted.

#### `apiBase`

```ts theme={null}
apiBase?: string;
```

Base URL for the OpenAI-compatible proxy. Falls back to the `BUB_API_BASE` env var if omitted.

#### `skills`

```ts theme={null}
skills?: SkillSpec[];
```

Skills to install into the Sandbox (a local directory/file, or a repo with an optional pinned ref and an optional enabled subset).
They land in `.agents/skills/<name>/`, with a discovery instruction written into AGENTS.md (bub has no native Skill loading mechanism).

#### `pythonPlugins`

```ts theme={null}
pythonPlugins?: PythonPluginSpec[];
```

Extra Python packages to install into the bub tool environment; on each Sandbox setup they go into `uv tool install … --with <pkg>`.
The normalized package list is part of the install checkpoint key: two agent variants with different plugin sets never reuse the same
install checkpoint (otherwise the second variant would silently inherit the first one's environment).

#### `postSetup`

```ts theme={null}
postSetup?: SandboxHook[];
```

User hooks that run, in array order, after install (reusing SandboxHook's narrow context): executed once installing
bub, installing Skills / Python packages, and writing the manifest are all done. A thrown error counts as an
infrastructure error (attempt errored).
See "Running scripts after install" in docs/feature/adapters/library/coding-agent-extensions.md.

#### `preTeardown`

```ts theme={null}
preTeardown?: SandboxHook[];
```

The teardown hook paired with `postSetup`: runs in reverse of `postSetup`'s ordering, before the agent's own teardown
step (a LIFO mirror — `postSetup` runs after the agent installs, `preTeardown` runs before the agent tears down), and
fires if and only if `postSetup`'s point in time was reached. A thrown error counts as an infrastructure error, folded
into a `teardown-failed` diagnostic by the teardown phase.
See "Running scripts after install" in docs/feature/adapters/library/coding-agent-extensions.md.

## `uiMessageStreamAgent`: built-in non-invasive adapter for AI SDK apps (with HITL)

`uiMessageStreamAgent` (exported from `niceeval/adapter`) talks non-invasively to an HTTP endpoint speaking the [UI Message Stream protocol](https://ai-sdk.dev/docs/ai-sdk-ui/stream-protocol) (the standard SSE of AI SDK `useChat` backends) — it only `fetch`es, never imports app code, so it connects wherever the app is deployed:

```ts theme={null}
import { uiMessageStreamAgent } from "niceeval/adapter";

export default uiMessageStreamAgent({
  name: "my-assistant",
  url: "https://my-app.example.com/api/chat",
  body: (ctx) => ({ model: ctx.model }),   // When the app supports request-level model selection, model comparison needs zero changes
});
```

Capabilities it handles for you:

* **Send/receive + event stream**: SSE frames are reduced by the `ai` package's official framework-agnostic reducer `readUIMessageStream` (the same one inside `useChat`); tool calls/results/message text are built directly from message parts — **the app is not required to wire up OTel**.
* **Session continuation**: the protocol is server-stateless, "client carries the full history" — the factory stores the whole `UIMessage[]` in an adapter-private typed session slot and replays it verbatim each turn; a new conversation line (after `t.newSession()`) gets an empty history.
* **HITL**: AI SDK v7 tool approval (tools with `needsApproval: true`) maps natively — when a part stops at `approval-requested`, the whole turn is `status: "waiting"` + `input.requested`; `t.respond("approve" / "deny")` is translated into `approval-responded`, rewriting that part in place and resending `messages` verbatim to trigger the server to continue (identical protocol behavior to a real frontend's `addToolApprovalResponse()` + `sendMessage()`; there is no separate approve endpoint). Denied calls land in the event stream as `rejected`, and by default carry a "do not retry" reason (`denyReason` can override) — without one, models often resend the very same call.
* **usage / waterfall**: UI Message Stream protocol frames carry no usage, so usage assertions like `t.maxTokens` have no data by default on this built-in (an app putting usage into message metadata is the app's own protocol extension). The waterfall is a separate matter: when the app has OTel instrumentation (e.g. the official `@ai-sdk/otel`), send spans to [NiceEval](https://niceeval.com/) per [OTel integration](/docs/tutorials/connect-otel) and `niceeval view` gets a full waterfall — spans only feed the waterfall, never assertions.

You need `ai` installed in the eval project (optional peer dependency; the protocol reducer comes from it). Full runnable example: [`examples/zh/tier1/ai-sdk-v7`](https://github.com/CorrectRoadH/niceeval/tree/main/examples/zh/tier1/ai-sdk-v7).

Full parameters of `uiMessageStreamAgent(options)` (`UiMessageStreamAgentOptions`):

#### `name`

```ts theme={null}
name?: string;
```

Agent name (identity for reports / result aggregation). Defaults to "ui-message-stream".

#### `url`

```ts theme={null}
url: string | ((ctx: AgentContext) => string | Promise<string>);
```

The chat endpoint of the system under test (a full URL, pointing wherever the app is deployed); the function form is resolved every turn.

#### `headers`

```ts theme={null}
headers?: Record<string, string> | ((ctx: AgentContext) => Record<string, string>);
```

Additional request headers (auth, etc.); `ctx.telemetry.headers` (traceparent) is always merged in automatically.

#### `body`

```ts theme={null}
body?: (ctx: AgentContext) => Record<string, JsonValue | undefined>;
```

Fields merged into the request body besides `messages`, e.g. `(ctx) => ({ model: ctx.model })` (undefined fields are dropped automatically during serialization).

#### `denyReason`

```ts theme={null}
denyReason?: string;
```

The reason carried with `approval-responded` when an approval is denied. The app/SDK passes it to the model as the tool result
text — spelling out "do not retry" measurably lowers the chance the model resends the very same call (observed empirically).

#### `settleMs`

```ts theme={null}
settleMs?: number;
```

How long to wait after the stream ends before returning (milliseconds), giving the app's observability export (e.g. a BatchSpanProcessor) time to flush.

#### `tracing`

```ts theme={null}
tracing?: AgentTracing;
```

How to deliver the endpoint when the app has OTel (for the waterfall); the event stream does not depend on it.

#### `spanMapper`

```ts theme={null}
spanMapper?: SpanMapper;
```

The span-normalizing function when the app has OTel (for the waterfall); the event stream does not depend on it.

## OpenAI-compatible response converters: `turnFromChatCompletion` / `turnFromResponses`

`turnFromChatCompletion(res)` / `turnFromResponses(res)` (exported from `niceeval/adapter`) map OpenAI's two response shapes — Chat Completions and Responses — from the whole response into a `Turn` with zero manual mapping: this is not limited to the official OpenAI API — any service that claims compatibility with either protocol shape works.

```ts theme={null}
import { completeEvidenceCoverage, defineAgent, turnFromChatCompletion } from "niceeval/adapter";

export default defineAgent({
  name: "my-openai-compat-agent",
  evidenceCoverage: completeEvidenceCoverage,
  async send(input) {
    const res = await client.chat.completions.create({ messages: [...], tools });
    return turnFromChatCompletion(res);
  },
});
```

The two shapes differ in how trustworthy negative assertions (`notCalledTool` and friends) are: Chat Completions does not guarantee "response = complete process" (the app may run the whole tool loop server-side and only hand you the final answer), so a negative assertion can only mean "not observed," not "did not happen"; the Responses protocol's contract guarantees the `output` array records everything the model decided to do this turn (including every `function_call`), so negative assertions there are trustworthy. The `Turn` shape the two converters produce is identical — this difference only affects how you interpret negative assertions.

## SDK event stream converters: `createClaudeSdkEventStream` / `createPiAgentEventStream` / `createCodexThreadEventStream`

Each agent SDK's streaming protocol is a generic protocol defined by the SDK, not some app's private format — the native-frame-to-standard-event mapping knowledge lives in the official package (exported from `niceeval/adapter`), so when writing your own non-invasive adapter only the **transport glue** remains (which endpoint the app puts the stream on, which endpoint approvals go through). Driving the loop frame by frame is itself an official piece (`driveFrameStream`, see the next section), so no hand-written `for` loop:

```ts theme={null}
import { createSessionSlot, sseJsonFrames, createClaudeSdkEventStream, driveFrameStream } from "niceeval/adapter";

const frames = sseJsonFrames<SDKMessage>(res.body);   // Standard SSE → per-frame JSON
const stream = createClaudeSdkEventStream();               // SDKMessage → standard events
return driveFrameStream(frames, stream, ctx);         // Feed stream.add() frame by frame, summarize into a Turn
```

* **`createClaudeSdkEventStream`** (Claude Agent SDK `SDKMessage`): `assistant` text/tool\_use blocks, `user` tool\_result blocks, `system`/`permission_denied` (→ `rejected`), `result` usage; `markRejected()` registers denied calls.
* **`createPiAgentEventStream`** (pi-agent-core `AgentEvent`): text/thinking/usage from `message_end`, tool pairs from `tool_execution_start/end`.
* **`createCodexThreadEventStream`** (Codex SDK `ThreadEvent`): message-class frames from `agent_message`/`reasoning`, tool items (`command_execution` / `mcp_tool_call` / `file_change` / `web_search` → paired tool `operation.started` + `operation.finished`), usage from `turn.completed`, error frames. The waterfall needs the codex CLI's native OTLP (the `[otel]` block in config.toml) sending spans to [NiceEval](https://niceeval.com/), normalized with the official `mapCodexSpans`.

Runnable references: the claude-sdk / pi-sdk / codex-sdk adapters under [`examples/zh/tier1`](https://github.com/CorrectRoadH/niceeval/tree/main/examples/zh/tier1) are exactly these three converters + `driveFrameStream` + transport glue.

## Generic "assembly" pieces: `driveFrameStream` / `deltaStream` + `ctx.session`

Inside a hand-written send, only three truly independent parts exist: transport (how to send), reduce (raw data → events, handled by the converters above), and orchestration (session continuation + HITL pause/resume). The third part is protocol-agnostic, pure control-flow pattern — session continuation (`id`/`capture`) and adapter-private typed slots hang directly off `ctx.session` (`AgentSession`, see [Adapter concept](/docs/explanation/adapter#context-agentcontext)). Create a slot once with `createSessionSlot<T>()`, then use `set(slot, value)` and `take(slot)` for a paused scene. What actually remains reusable across protocols is just two official pieces:

| Piece                                              | What it solves                                                                                                                                                                                                                                                                                                                                                            |
| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `driveFrameStream(cursor, reducer, ctx, onFrame?)` | Feeds the reducer frame by frame, handles transport frames, detects the HITL pause signal — the loop itself collapsed into one function. On pause the `cursor` is not closed: store it with `ctx.session.set(pendingSlot, cursor)` inside `onFrame`, and on the answer turn `ctx.session.take(pendingSlot)` retrieves it and keeps reading, without issuing a new request |
| `deltaStream(spec)`                                | Generic accumulator for token-by-token / per-argument deltas — for protocols with "no whole-chunk landing frames" like raw OpenAI/Anthropic streaming APIs or your own hand-rolled token-by-token backend; declare "which operation this frame belongs to" (text delta / tool-argument delta / finalize) and it manages assembly timing itself                            |

```ts theme={null}
// Create this once at adapter module scope.
const pendingSlot = createSessionSlot<Pending>("my-app/pending");

// First thing in send: check for a held scene (HITL answer turn)
const pending = ctx.session.take(pendingSlot);
if (pending) {
  // ...match verdicts from input.responses, hand the approval back to the app...
  return driveFrameStream(pending.cursor, pending.stream, ctx, onFrame);
}

// Normal turn: session continuation via ctx.session.id / capture; on pause, a typed slot stores the cursor
return driveFrameStream(cursor, stream, ctx, (frame, derived) => {
  ctx.session.capture(stream.sessionId);
  const gated = derived.find((event) =>
    event.type === "operation.started" &&
    event.operation.kind === "tool" &&
    event.operation.name === GATED_TOOL_NAME
  );
  if (gated?.type === "operation.started") {
    ctx.session.set(pendingSlot, { cursor, stream, toolUseId: gated.operationId });
    return { pause: { id: gated.operationId, action: GATED_TOOL_NAME, options: [{ id: "approve" }, { id: "deny" }] } };
  }
});
```

The three Tier 1 examples (claude-sdk and pi-sdk use `driveFrameStream` plus typed session slots; codex-sdk uses only `driveFrameStream`) are built on these pieces — the adapters are down to transport plus the "does this frame need extra handling" decision in `onFrame`, with no hand-written loops or module-level Maps.

## `turnFromAiSdk`: AI SDK result → event stream converter

`turnFromAiSdk` (exported from `niceeval/adapter`) is used inside adapters you write yourself (e.g. server-side direct construction for an HTTP web agent, see `examples/zh/ai-sdk/`). It maps an AI SDK `generateText` / `streamText` result into `{ events, usage, status }` — exact `toolCallId` pairing, order-faithful, `tool-error` mapped to a failed tool `operation.finished`, usage aggregation (v4 / v5 / v7 field drift all covered) — laid straight into the `Turn`:

```ts theme={null}
import { completeEvidenceCoverage, defineAgent, turnFromAiSdk } from "niceeval/adapter";

export default defineAgent({
  name: "my-ai-sdk-agent",
  evidenceCoverage: completeEvidenceCoverage,
  async send(input) {
    const result = await generateText({ model, tools, prompt: input.text });
    return { ...turnFromAiSdk(result), data: result.text };
  },
});
```

The `status` comes from the converter: `"waiting"` when there is a tool approval pending human review (with an `input.requested` event attached), otherwise `"completed"`. Session continuation (multi-turn resume), handing HITL verdicts back, and tracing depend on how you write your own `send`.

## How to choose

* The system under test is an AI SDK app (`useChat` backend): the built-in `uiMessageStreamAgent` connects non-invasively with zero mapping (including HITL).
* The system under test is another agent system (HTTP / gRPC service): connect non-invasively and hand-write the event mapping (official converters cover most of it) — all five examples under [`examples/zh/tier1`](https://github.com/CorrectRoadH/niceeval/tree/main/examples/zh/tier1) take this path; if the app already has OTel instrumentation, wire it up for a bonus waterfall.
* You need the call waterfall (the trace in `niceeval view`): all three of `claude-code` / `codex` / `bub` emit OTLP themselves (claude-code via the CLI's beta telemetry; its spans carry structure and timing only); adapters you write yourself just declare `tracing` (see the OTel guide).
* You need human approval / multi-step confirmation (HITL): `uiMessageStreamAgent` supports it natively (AI SDK v7 tool approval); none of the three sandbox agents do; for other systems write your own adapter, assembled from `driveFrameStream` plus typed session slots (the pi-sdk / claude-sdk Tier 1 examples have ready-made patterns).
* You want to run a coding agent that edits code, inspect diffs, and judge tool calls: `claude-code` / `codex` / `bub` (send/receive + event stream + session continuation + workspace + sandbox).
* Your backend protocol is home-grown with no official SDK converter: do not hand-write send from scratch — pick a reduce shape (a small mapping for whole-chunk landing, `deltaStream` for token-by-token deltas), use `ctx.session.id`/`capture` for session continuation, add a typed session slot if you have HITL, and the assembly is your send.

## Related reading

* [Connect your agent](/docs/tutorials/connect-your-agent) — how to implement each capability in your own adapter and which assertions it maps to.
* [Sandbox Agent](/docs/tutorials/sandbox-agent) — how to run the built-in sandbox agents and write your own.
* [defineAgent reference](/docs/reference/define-agent) — full parameters of `defineAgent` / `defineSandboxAgent`.
* [OTel integration](/docs/tutorials/connect-otel) — send your app's spans to [NiceEval](https://niceeval.com/) in exchange for the call waterfall in `niceeval view`.