Repository navigation
feat: ledger client reads the records of a flow - #2190
Conversation
de189fc to
40f5736
Compare
|
One should-fix. Document in the |
A component reads the interactions, autonomous agent activity, guardrail decisions and evaluations of a flow, one page at a time, in the order they were recorded.
An LLM judge reads the interactions, activity and guardrail decisions of a flow as text, without the verdicts of other evaluators and without a repeated system message.
The ledger records no subtask and no worker for it, so the javadoc and the transcript count delegations instead of subtasks.
… interaction A report-only decision and a failure of a fail-open guardrail let the interaction continue, so a judge must not read them as a block.
A judge matches an interaction to the activity of its agent instance and task, also when an agent delegates to several workers of one component.
A seed writes the seed order before the record, so the read never sorts a record without its order.
A real position is a sequence number in the flow log, not an index of the records, so a test must not compute one.
…w run A test kit flow read no longer throws when a workflow evaluation is seeded, and the workflow evaluation spec compiles against the SPI flow read.
40f5736 to
584b988
Compare
|
fixed the comment, and rebased |
|
🤖 says... [should-fix]
In a flow with more than one agent, a judge cannot tell whose interaction a decision belongs to. The runtime records a decision while the interaction runs, and records the interaction only when it ends. So a decision always comes before its own interaction block, and often sits right above the block of another agent. The change's own fixture shows this. In A judge reads this DENY as a decision on [nit]
A position is an index into the records sorted by timestamp. A record seeded after a read, with an older timestamp, moves each record already read one index later. The read from [nit]
The javadoc says A page whose records were all dropped is empty, but its |
|
🤖 Thanks for the review, all points fixed:
|
Read records of an autonomous agent flow.
SDK side of https://github.com/lightbend/akka-runtime/pull/6065
🤖
An evaluator of an autonomous agent's interaction can now read the whole flow of that interaction. Before, it saw
only the interaction itself.
LedgerClient.getFlowRecordsreturns the interactions, the activity of the autonomousagents, the guardrail decisions and the evaluations of a flow. It returns them in the order they were recorded, in
pages of at most 100. A page carries the position of the next page and a flag that says whether the flow held more
records. A later read from the position of the last page returns the records recorded after it. The activity holds what the interaction alone does not show: whether the task completed, how
many iterations it used, whether a result was rejected, and the delegations and handoffs.
FlowRecord.transcriptrenders the records as text for an LLM judge. Each interaction names its agent instance and task, so the judge can
match it to the activity lines. A guardrail line says when the decision only reported and did not stop the
interaction. A repeated system message is written once per agent, and the verdicts of other evaluators are left out.
The activity events use the names and field order of
Notification. Each event'skind()is the name in the storedrecord, and a test checks it against the SPI. A record class or an event class that a newer runtime adds does not
break an older SDK: the record is left out of the page and the event reads as
Otherwith its kind. An evaluationoutcome or a message content that the SDK does not know reads as the runtime reads one from storage: the outcome as
inconclusive, with a reason that says so, and the content is left out. This also applies to
getEvaluation,getEvaluationsandgetInteraction. A flow page leaves out an evaluation of a subject that is not an interaction,which
EvaluationRecordcannot hold.TestLedgerClientserves the same read from seeded records. Integration tests read the flow of a coordinator that delegates to an
autonomous worker, and of an agent that delegates to a request-based agent.