Skip to content

feat: ledger client reads the records of a flow - #2190

Merged
patriknw merged 26 commits into
mainfrom
wip-ledger-flow-read-patriknw
Nov 10, 2026
Merged

patriknw merged 26 commits into
mainfrom
wip-ledger-flow-read-patriknw

Conversation

@patriknw

@patriknw patriknw commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Read records of an autonomous agent flow.

SDK side of https://github.com/lightbend/akka-runtime/pull/6065


🤖

An evaluator of an autonomous agent's interaction can now read the whole flow of that interaction. Before, it saw
only the interaction itself. LedgerClient.getFlowRecords returns the interactions, the activity of the autonomous
agents, the guardrail decisions and the evaluations of a flow. It returns them in the order they were recorded, in
pages of at most 100. A page carries the position of the next page and a flag that says whether the flow held more
records. A later read from the position of the last page returns the records recorded after it. The activity holds what the interaction alone does not show: whether the task completed, how
many iterations it used, whether a result was rejected, and the delegations and handoffs. FlowRecord.transcript
renders the records as text for an LLM judge. Each interaction names its agent instance and task, so the judge can
match it to the activity lines. A guardrail line says when the decision only reported and did not stop the
interaction. A repeated system message is written once per agent, and the verdicts of other evaluators are left out.

The activity events use the names and field order of Notification. Each event's kind() is the name in the stored
record, and a test checks it against the SPI. A record class or an event class that a newer runtime adds does not
break an older SDK: the record is left out of the page and the event reads as Other with its kind. An evaluation
outcome or a message content that the SDK does not know reads as the runtime reads one from storage: the outcome as
inconclusive, with a reason that says so, and the content is left out. This also applies to getEvaluation,
getEvaluations and getInteraction. A flow page leaves out an evaluation of a subject that is not an interaction,
which EvaluationRecord cannot hold. TestLedgerClient
serves the same read from seeded records. Integration tests read the flow of a coordinator that delegates to an
autonomous worker, and of an agent that delegates to a request-based agent.

@github-actions github-actions Bot added the documentation documentation related label Nov 5, 2026
@patriknw
patriknw force-pushed the wip-ledger-flow-read-patriknw branch from de189fc to 40f5736 Compare November 6, 2026 13:28
@aludwiko

aludwiko commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

One should-fix. Document in the getFlowRecords javadoc that unknown record classes and non-interaction evaluations are dropped from a page.

A component reads the interactions, autonomous agent activity, guardrail decisions and evaluations of a flow, one page at a time, in the order they were recorded.
An LLM judge reads the interactions, activity and guardrail decisions of a flow as text, without the verdicts of other evaluators and without a repeated system message.
The ledger records no subtask and no worker for it, so the javadoc and the transcript count delegations instead of subtasks.
… interaction

A report-only decision and a failure of a fail-open guardrail let the interaction continue, so a judge must not read them as a block.
A judge matches an interaction to the activity of its agent instance and task, also when an agent delegates to several workers of one component.
A seed writes the seed order before the record, so the read never sorts a record without its order.
A real position is a sequence number in the flow log, not an index of the records, so a test must not compute one.
…w run

A test kit flow read no longer throws when a workflow evaluation is seeded, and the workflow evaluation spec compiles against the SPI flow read.
@patriknw
patriknw force-pushed the wip-ledger-flow-read-patriknw branch from 40f5736 to 584b988 Compare November 9, 2026 15:17
@patriknw

patriknw commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

fixed the comment, and rebased

@octonato

Copy link
Copy Markdown
Contributor

🤖 says...

[should-fix]
A guardrail line in the transcript does not name its agent

akka-javasdk/src/main/java/akka/javasdk/ledger/FlowRecord.java:185

guardrailDecisionLine writes the guardrail name, boundary, tool, decision and explanation. It leaves out the agent, although GuardrailDecisionRecord carries the agent component id, the session id and the interaction id.

In a flow with more than one agent, a judge cannot tell whose interaction a decision belongs to. The runtime records a decision while the interaction runs, and records the interaction only when it ends. So a decision always comes before its own interaction block, and often sits right above the block of another agent.

The change's own fixture shows this. In akka-javasdk/src/test/java/akka/javasdk/ledger/FlowRecordTest.java:164, a DENY recorded for coordinator-agent renders directly above the worker-agent interaction and names no agent. With parallel workers, a report-only DENY on worker-b renders right above the worker-a interaction:

Activity of agent coordinator, instance coordinator-1: started 2 delegations to [worker-a (worker-a-1), worker-b (worker-b-1)], subtasks [s1, s2]

Guardrail pii, boundary BEFORE_AGENT_RESPONSE, decision DENY, report only: response holds an email address

Interaction of agent worker-a, instance worker-a-1, task research (id s1), iteration 1:
Input: Find contacts
Response: Alice, alice@example.com

A judge reads this DENY as a decision on worker-a.

[nit]
The test kit repeats a record and skips another after a back-dated seed

akka-javasdk-testkit/src/main/java/akka/javasdk/testkit/TestLedgerClient.java:259

A position is an index into the records sorted by timestamp. A record seeded after a read, with an older timestamp, moves each record already read one index later.

The read from nextPosition then returns a record a second time and never returns the new one. This breaks the FlowPage promise that a read from nextPosition also returns records recorded later. Seed a1 at t=10, read (next is 2), seed a2 at t=5, then read from 2: the second read returns [a1] again.

[nit]
The nextPosition javadoc is wrong for a page the client emptied

akka-javasdk/src/main/java/akka/javasdk/ledger/FlowPage.java:15

The javadoc says nextPosition is the position the read started from when the page is empty. LedgerClientImpl.toFlowPage drops workflow evaluations and records of unknown classes, but keeps the runtime nextPosition, which points past the dropped records.

A page whose records were all dropped is empty, but its nextPosition is not the start position. A caller who trusts the javadoc might keep the start position on an empty page, and read the same runtime page again and again while hasMore is true.

@patriknw

Copy link
Copy Markdown
Contributor Author

🤖 Thanks for the review, all points fixed:

  • Records left out of a page (584b9887a): the getFlowRecords javadoc now names them, an unknown record class and an evaluation of a workflow run. A page can then be short or empty while hasMore is true, so read on hasMore.
  • Guardrail line names its agent (613f74ee1): Guardrail pii for agent worker-b, instance worker-b-1, .... The instance comes from the decision's interaction when it is in the list. Your parallel-worker example is now a test.
  • Test kit positions (9c662c8c0): a flow is now in seed order, like the runtime's write order, and a position is the seed number. A later seed never moves a position, whatever its timestamp. Your example is now a test.
  • nextPosition of an emptied page (a3e9fcc12): the javadoc says it is after the last record the read reached, also one the client left out, so it can be past the start. A new test covers it.

@patriknw
patriknw merged commit 7e608be into main Nov 10, 2026
38 checks passed
@patriknw
patriknw deleted the wip-ledger-flow-read-patriknw branch November 10, 2026 15:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation documentation related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants