> ## Documentation Index
> Fetch the complete documentation index at: https://niceeval.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Standard event stream reference

> The ten StreamEvent variants: what each one means, when adapters emit it, and which assertions consume it. Producing this stream is the adapter's core job.

An adapter's `send` returns a `Turn`, and its `events: StreamEvent[]` is the **only data source for assertions**: `t.calledTool()`, `t.reply`, `toolOrder`, `noFailedActions`, and more all read from this stream. Once your adapter translates "what the agent did this turn" into this stream, the whole assertion surface becomes available.

## Turn: the return value of `send`

```ts theme={null}
interface Turn {
  events: StreamEvent[];                          // this turn's events, in real order
  data?: JsonValue;                               // structured output → outputEquals / outputMatches
  status: "completed" | "failed" | "waiting";     // waiting = paused for human input (HITL)
  usage?: Usage;                                  // → maxTokens / maxCost / cost reports, full fields below
}
```

The meaning of `data` is "this turn's structured product": only fill it when **the application's answer itself is a structured object** (extraction, classification, form filling); `outputEquals` / `outputMatches` read from it. Applications that only reply with text should leave it unset — do not copy the raw response body or `message` text into it to pad it out, and conversely do not serialize structured output into `events`. Include `usage` when you have it, and leave it unset when you don't — **never invent numbers**.

The full field set of `usage` (the `Usage` type):

#### `inputTokens`

```ts theme={null}
inputTokens?: number;
```

Total input tokens at billing granularity (whatever the protocol reports; includes cache reads as-is, no conversion).

#### `outputTokens`

```ts theme={null}
outputTokens?: number;
```

Output (completion) token count.

#### `cacheReadTokens`

```ts theme={null}
cacheReadTokens?: number;
```

The portion of input served from cache hits; same accounting basis as `inputTokens` (omitted means the agent does not report this).

#### `cacheCreationTokens`

```ts theme={null}
cacheCreationTokens?: number;
```

Tokens written to create a prompt cache entry (omitted means the agent does not report this).

#### `reasoningTokens`

```ts theme={null}
reasoningTokens?: number;
```

Reasoning (thinking) token count, present only when the protocol actually reports it (omitted means the agent does not report this).

#### `requests`

```ts theme={null}
requests?: number;
```

The actual number of model requests that occurred. Omitted when the protocol doesn't provide a request count — never padded to 1.

#### `costUSD`

```ts theme={null}
costUSD?: number;
```

The actual dollar cost measured by the gateway/adapter (can only be brought back explicitly via `Turn.usage.costUSD`; it is
never inferred from token usage or an OTel span). This is a separate fact from the top-level `estimatedCostUSD` (price-table
estimate): when present, it takes priority over the cost estimated from the price table (`defineConfig({ pricing })`) — see
the fallback order `usage.costUSD ?? estimateCost(...)` in `estimateCost`.

## `StreamEvent` variants at a glance

The ten variants of `StreamEvent`, listed field by field (see the "Event table" and "Event details" sections below for the assertions / usage details that consume them):

#### `message`

```ts theme={null}
{ type: "message"; role: "assistant" | "user"; text: string; loc?: SourceLoc }
```

#### `operation.started`

```ts theme={null}
{
  type: "operation.started";
  operationId: string;
  operation:
    | { kind: "tool"; name: string; input: JsonValue; tool?: ToolName }
    | { kind: "subagent"; name: string; remoteUrl?: string };
}
```

#### `operation.finished`

```ts theme={null}
{
  type: "operation.finished";
  operationId: string;
  kind: "tool";
  output?: JsonValue;
  status: "completed" | "failed" | "rejected";
}
```

#### `operation.finished`

```ts theme={null}
{
  type: "operation.finished";
  operationId: string;
  kind: "subagent";
  output?: JsonValue;
  status: "completed" | "failed";
}
```

#### `skill.loaded`

```ts theme={null}
{ type: "skill.loaded"; skill: string; operationId?: string }
```

#### `input.requested`

```ts theme={null}
{ type: "input.requested"; request: InputRequest }
```

#### `thinking`

```ts theme={null}
{ type: "thinking"; text: string }
```

#### `compaction`

```ts theme={null}
{ type: "compaction"; reason?: string }
```

#### `error`

```ts theme={null}
{ type: "error"; message: string }
```

## Event table

| Event                                     | Meaning                                   | Consumed by                                                                                                           |
| ----------------------------------------- | ----------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| `message`                                 | the agent (or user) said something        | `t.reply`, `messageIncludes`, judge inputs                                                                            |
| `operation.started` (`kind: "tool"`)      | a tool call started                       | `calledTool`, `toolOrder`, `maxToolCalls`, `notCalledTool`                                                            |
| `operation.finished` (`kind: "tool"`)     | the result of that tool call              | `calledTool` status matching, `noFailedActions`                                                                       |
| `operation.started` (`kind: "subagent"`)  | delegated to a subagent                   | `calledSubagent`                                                                                                      |
| `operation.finished` (`kind: "subagent"`) | the subagent returned                     | `calledSubagent` status, `noFailedActions`                                                                            |
| `input.requested`                         | paused and waiting for human input (HITL) | `t.parked()`, `t.requireInputRequest()`                                                                               |
| `thinking`                                | reasoning text                            | `event("thinking")`, viewer display                                                                                   |
| `compaction`                              | context got compacted                     | `event("compaction")` (the adapter's parser emitting this event is enough to assert on it — no declaration is needed) |
| `error`                                   | the turn hit an error                     | `event("error")`, viewer display                                                                                      |

Any event can also be consumed by the generic assertions: `event(type)` / `notEvent(type)` / `eventOrder(types)` / `eventsSatisfy(label, predicate)`.

## Event details

### `message` — what was said

```ts theme={null}
{ type: "message", role: "assistant" | "user", text: string }
```

Emit one `message` with `role: "assistant"` for each chunk of assistant text. **Tool results are not assistant messages** — do not wrap tool output as `message`, or `t.reply` will read the wrong thing. `message` events for user input are recorded automatically by [NiceEval](https://niceeval.com/); the adapter does not need to emit them.

### Tool `operation.started` / `operation.finished` — which tool was called, and what happened

```ts theme={null}
{ type: "operation.started", operationId: string,
  operation: { kind: "tool", name: string, input: JsonValue, tool?: ToolName } }
{ type: "operation.finished", operationId: string, kind: "tool",
  output?: JsonValue, status: "completed" | "failed" | "rejected" }
```

* Pair each tool `operation.started` with a tool `operation.finished` that has the **same `operationId`** — this is what keeps concurrent calls from getting mixed up. Use the explicit id your agent's response gives you (AI SDK's `toolCallId`, Anthropic's `tool_use.id`) directly; only synthesize one by order as a last resort when there truly isn't one.
* Fill `status` truthfully: an actual tool execution failure is `"failed"` (`noFailedActions()` fires on it); **a human denial is `"rejected"`** (`noFailedActions()` still passes, and `calledTool(..., { status: "rejected" })` can assert on it precisely). These are two different things — don't conflate them.
* Use the tool's original name for `name`.

### Subagent `operation.started` / `operation.finished` — who was delegated to

```ts theme={null}
{ type: "operation.started", operationId: string,
  operation: { kind: "subagent", name: string, remoteUrl?: string } }
{ type: "operation.finished", operationId: string, kind: "subagent",
  output?: JsonValue, status: "completed" | "failed" }
```

Emit this pair when the system under test delegates a task to a subagent (and waits for it to return); the `operationId` pairing rule is the same as above. This feeds assertions like `calledSubagent("researcher")`.

### `input.requested` — paused for human input (HITL)

```ts theme={null}
{ type: "input.requested", request: {
    id?: string,
    action?: string,        // which action it's paused on (e.g. a tool name)
    input?: JsonValue,      // that action's arguments
    prompt?: string,        // the question posed to the human
    options?: { id: string, label?: string }[],   // choices (approve / deny…)
} }
```

When the agent pauses for a person, emit one event per pending question, and the whole turn's `status` returns `"waiting"`. The filter in `t.requireInputRequest(filter)` matches this `request` field by field — **fill in as many fields as you can**, or the eval side won't be able to filter it. See the [HITL section of the connect-your-agent tutorial](/docs/tutorials/connect-your-agent).

### `thinking` / `compaction` / `error`

```ts theme={null}
{ type: "thinking", text: string }
{ type: "compaction", reason?: string }   // context compaction; the adapter's parser emitting it is enough to assert on it — no declaration layer
{ type: "error", message: string }
```

Emit them when you have them; don't fabricate them when you don't. `compaction` mostly comes from coding-agent CLIs (automatic compaction when context fills up).

## Three mapping rules

1. **Order is fact**: events are laid out in the order they actually happened. `toolOrder` / `eventOrder` match by subsequence, so a wrong order makes the assertion misleading.
2. **Pair `operationId`s**: every started operation needs a finished operation of the same `kind` with the same id; missing half of the pair makes `calledTool(..., { status })` fail to match.
3. **Completeness has no declaration layer — it is proven by the source**: with official converters (`createClaudeSdkEventStream`, `turnFromAiSdk`, etc.), the return value itself guarantees **every** tool call is in the stream; with manual mapping, completeness depends entirely on whether your mapping code covers everything. When only part of the stream is emitted, negative assertions like `notCalledTool` / `maxToolCalls` pass silently — with no error, which is harder to notice than an outright failure. Confirm your mapping code covers every tool call before relying on manual mapping — see the [capabilities reference](/docs/reference/capabilities).

## A complete mapping example

When the agent response contains step records, the mapping is a small loop:

```ts theme={null}
import type { StreamEvent } from "niceeval";

function toStreamEvents(body: MyAgentResponse): StreamEvent[] {
  const events: StreamEvent[] = [];
  for (const step of body.steps) {
    if (step.type === "tool_call") {
      events.push({
        type: "operation.started",
        operationId: step.id,
        operation: { kind: "tool", name: step.tool, input: step.args },
      });
      events.push({
        type: "operation.finished",
        operationId: step.id,
        kind: "tool",
        output: step.result,
        status: step.error ? "failed" : "completed",
      });
    }
    if (step.type === "text") events.push({ type: "message", role: "assistant", text: step.text });
  }
  return events;
}
```

## Related reading

* [Connect your agent](/docs/tutorials/connect-your-agent) — the from-scratch integration tutorial.
* [Capabilities](/docs/reference/capabilities) — what it means to say "the event stream is complete."
* [Authoring evals](/docs/tutorials/authoring) — the full set of assertions that consume this stream.