Standard event stream reference

An adapter’s send returns a Turn, and its events: StreamEvent[] is the only data source for assertions: t.calledTool(), t.reply, toolOrder, noFailedActions, and more all read from this stream. Once your adapter translates “what the agent did this turn” into this stream, the whole assertion surface becomes available.

Turn: the return value of `send`

interface Turn {
  events: StreamEvent[];                          // this turn's events, in real order
  data?: JsonValue;                               // structured output → outputEquals / outputMatches
  status: "completed" | "failed" | "waiting";     // waiting = paused for human input (HITL)
  usage?: Usage;                                  // → maxTokens / maxCost / cost reports, full fields below
}

The meaning of data is “this turn’s structured product”: only fill it when the application’s answer itself is a structured object (extraction, classification, form filling); outputEquals / outputMatches read from it. Applications that only reply with text should leave it unset — do not copy the raw response body or message text into it to pad it out, and conversely do not serialize structured output into events. Include usage when you have it, and leave it unset when you don’t — never invent numbers. The full field set of usage (the Usage type):

`inputTokens`

inputTokens?: number;

Total input tokens at billing granularity (whatever the protocol reports; includes cache reads as-is, no conversion).

`outputTokens`

outputTokens?: number;

Output (completion) token count.

`cacheReadTokens`

cacheReadTokens?: number;

The portion of input served from cache hits; same accounting basis as inputTokens (omitted means the agent does not report this).

`cacheCreationTokens`

cacheCreationTokens?: number;

Tokens written to create a prompt cache entry (omitted means the agent does not report this).

`reasoningTokens`

reasoningTokens?: number;

Reasoning (thinking) token count, present only when the protocol actually reports it (omitted means the agent does not report this).

`requests`

requests?: number;

The actual number of model requests that occurred. Omitted when the protocol doesn’t provide a request count — never padded to 1.

`costUSD`

costUSD?: number;

The actual dollar cost measured by the gateway/adapter (can only be brought back explicitly via Turn.usage.costUSD; it is never inferred from token usage or an OTel span). This is a separate fact from the top-level estimatedCostUSD (price-table estimate): when present, it takes priority over the cost estimated from the price table (defineConfig({ pricing })) — see the fallback order usage.costUSD ?? estimateCost(...) in estimateCost.

`StreamEvent` variants at a glance

The ten variants of StreamEvent, listed field by field (see the “Event table” and “Event details” sections below for the assertions / usage details that consume them):

`message`

{ type: "message"; role: "assistant" | "user"; text: string; loc?: SourceLoc }

`operation.started`

{
  type: "operation.started";
  operationId: string;
  operation:
    | { kind: "tool"; name: string; input: JsonValue; tool?: ToolName }
    | { kind: "subagent"; name: string; remoteUrl?: string };
}

`operation.finished`

{
  type: "operation.finished";
  operationId: string;
  kind: "tool";
  output?: JsonValue;
  status: "completed" | "failed" | "rejected";
}

`operation.finished`

{
  type: "operation.finished";
  operationId: string;
  kind: "subagent";
  output?: JsonValue;
  status: "completed" | "failed";
}

`skill.loaded`

{ type: "skill.loaded"; skill: string; operationId?: string }

`input.requested`

{ type: "input.requested"; request: InputRequest }

`thinking`

{ type: "thinking"; text: string }

`compaction`

{ type: "compaction"; reason?: string }

`error`

{ type: "error"; message: string }

Event table

Event	Meaning	Consumed by
`message`	the agent (or user) said something	`t.reply`, `messageIncludes`, judge inputs
`operation.started` (`kind: "tool"`)	a tool call started	`calledTool`, `toolOrder`, `maxToolCalls`, `notCalledTool`
`operation.finished` (`kind: "tool"`)	the result of that tool call	`calledTool` status matching, `noFailedActions`
`operation.started` (`kind: "subagent"`)	delegated to a subagent	`calledSubagent`
`operation.finished` (`kind: "subagent"`)	the subagent returned	`calledSubagent` status, `noFailedActions`
`input.requested`	paused and waiting for human input (HITL)	`t.parked()`, `t.requireInputRequest()`
`thinking`	reasoning text	`event("thinking")`, viewer display
`compaction`	context got compacted	`event("compaction")` (the adapter’s parser emitting this event is enough to assert on it — no declaration is needed)
`error`	the turn hit an error	`event("error")`, viewer display

Any event can also be consumed by the generic assertions: event(type) / notEvent(type) / eventOrder(types) / eventsSatisfy(label, predicate).

Event details

`message` — what was said

{ type: "message", role: "assistant" | "user", text: string }

Emit one message with role: "assistant" for each chunk of assistant text. Tool results are not assistant messages — do not wrap tool output as message, or t.reply will read the wrong thing. message events for user input are recorded automatically by NiceEval; the adapter does not need to emit them.

Tool `operation.started` / `operation.finished` — which tool was called, and what happened

{ type: "operation.started", operationId: string,
  operation: { kind: "tool", name: string, input: JsonValue, tool?: ToolName } }
{ type: "operation.finished", operationId: string, kind: "tool",
  output?: JsonValue, status: "completed" | "failed" | "rejected" }

Pair each tool operation.started with a tool operation.finished that has the same operationId — this is what keeps concurrent calls from getting mixed up. Use the explicit id your agent’s response gives you (AI SDK’s toolCallId, Anthropic’s tool_use.id) directly; only synthesize one by order as a last resort when there truly isn’t one.
Fill status truthfully: an actual tool execution failure is "failed" (noFailedActions() fires on it); a human denial is "rejected" (noFailedActions() still passes, and calledTool(..., { status: "rejected" }) can assert on it precisely). These are two different things — don’t conflate them.
Use the tool’s original name for name.

Subagent `operation.started` / `operation.finished` — who was delegated to

{ type: "operation.started", operationId: string,
  operation: { kind: "subagent", name: string, remoteUrl?: string } }
{ type: "operation.finished", operationId: string, kind: "subagent",
  output?: JsonValue, status: "completed" | "failed" }

Emit this pair when the system under test delegates a task to a subagent (and waits for it to return); the operationId pairing rule is the same as above. This feeds assertions like calledSubagent("researcher").

`input.requested` — paused for human input (HITL)

{ type: "input.requested", request: {
    id?: string,
    action?: string,        // which action it's paused on (e.g. a tool name)
    input?: JsonValue,      // that action's arguments
    prompt?: string,        // the question posed to the human
    options?: { id: string, label?: string }[],   // choices (approve / deny…)
} }

When the agent pauses for a person, emit one event per pending question, and the whole turn’s status returns "waiting". The filter in t.requireInputRequest(filter) matches this request field by field — fill in as many fields as you can, or the eval side won’t be able to filter it. See the HITL section of the connect-your-agent tutorial.

`thinking` / `compaction` / `error`

{ type: "thinking", text: string }
{ type: "compaction", reason?: string }   // context compaction; the adapter's parser emitting it is enough to assert on it — no declaration layer
{ type: "error", message: string }

Emit them when you have them; don’t fabricate them when you don’t. compaction mostly comes from coding-agent CLIs (automatic compaction when context fills up).

Three mapping rules

Order is fact: events are laid out in the order they actually happened. toolOrder / eventOrder match by subsequence, so a wrong order makes the assertion misleading.
Pair operationIds: every started operation needs a finished operation of the same kind with the same id; missing half of the pair makes calledTool(..., { status }) fail to match.
Completeness has no declaration layer — it is proven by the source: with official converters (createClaudeSdkEventStream, turnFromAiSdk, etc.), the return value itself guarantees every tool call is in the stream; with manual mapping, completeness depends entirely on whether your mapping code covers everything. When only part of the stream is emitted, negative assertions like notCalledTool / maxToolCalls pass silently — with no error, which is harder to notice than an outright failure. Confirm your mapping code covers every tool call before relying on manual mapping — see the capabilities reference.

A complete mapping example

When the agent response contains step records, the mapping is a small loop:

import type { StreamEvent } from "niceeval";

function toStreamEvents(body: MyAgentResponse): StreamEvent[] {
  const events: StreamEvent[] = [];
  for (const step of body.steps) {
    if (step.type === "tool_call") {
      events.push({
        type: "operation.started",
        operationId: step.id,
        operation: { kind: "tool", name: step.tool, input: step.args },
      });
      events.push({
        type: "operation.finished",
        operationId: step.id,
        kind: "tool",
        output: step.result,
        status: step.error ? "failed" : "completed",
      });
    }
    if (step.type === "text") events.push({ type: "message", role: "assistant", text: step.text });
  }
  return events;
}

Connect your agent — the from-scratch integration tutorial.
Capabilities — what it means to say “the event stream is complete.”
Authoring evals — the full set of assertions that consume this stream.

API

Adapters

Results & Reports

CLI

Standard event stream reference

Turn: the return value of `send`

`inputTokens`

`outputTokens`

`cacheReadTokens`

`cacheCreationTokens`

`reasoningTokens`

`requests`

`costUSD`

`StreamEvent` variants at a glance

`message`

`operation.started`

`operation.finished`

`operation.finished`

`skill.loaded`

`input.requested`

`thinking`

`compaction`

`error`

Event table

Event details

`message` — what was said

Tool `operation.started` / `operation.finished` — which tool was called, and what happened

Subagent `operation.started` / `operation.finished` — who was delegated to

`input.requested` — paused for human input (HITL)

`thinking` / `compaction` / `error`

Three mapping rules

A complete mapping example

​Turn: the return value of send

​inputTokens

​outputTokens

​cacheReadTokens

​cacheCreationTokens

​reasoningTokens

​requests

​costUSD

​StreamEvent variants at a glance

​message

​operation.started

​operation.finished

​operation.finished

​skill.loaded

​input.requested

​thinking

​compaction

​error

​Event table

​Event details

​message — what was said

​Tool operation.started / operation.finished — which tool was called, and what happened

​Subagent operation.started / operation.finished — who was delegated to

​input.requested — paused for human input (HITL)

​thinking / compaction / error

​Three mapping rules

​A complete mapping example

​Related reading

Turn: the return value of `send`

`inputTokens`

`outputTokens`

`cacheReadTokens`

`cacheCreationTokens`

`reasoningTokens`

`requests`

`costUSD`

`StreamEvent` variants at a glance

`message`

`operation.started`

`operation.finished`

`operation.finished`

`skill.loaded`

`input.requested`

`thinking`

`compaction`

`error`

Event table

Event details

`message` — what was said

Tool `operation.started` / `operation.finished` — which tool was called, and what happened

Subagent `operation.started` / `operation.finished` — who was delegated to

`input.requested` — paused for human input (HITL)

`thinking` / `compaction` / `error`

Three mapping rules

A complete mapping example

Related reading