Skip to content

feat: ToolCallGuardrails on MCP tools - #1995

Merged
octonato merged 11 commits into
mainfrom
octonato/new-guardrails-for-mcp-sdk
Oct 13, 2026
Merged

octonato merged 11 commits into
mainfrom
octonato/new-guardrails-for-mcp-sdk

Conversation

@octonato

@octonato octonato commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot added the documentation documentation related label Oct 7, 2026
Base automatically changed from feature/governance to main October 7, 2026 07:46
@patriknw
patriknw force-pushed the octonato/new-guardrails-for-mcp-sdk branch from f380208 to 49558b0 Compare October 7, 2026 08:41
@patriknw

patriknw commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

I rebased this, worth noting:

origin match {
case SpiGuardrail.ToolOrigin.FunctionTool => new ToolCallGuardrail.ToolOrigin.FunctionTool()
case mcp: SpiGuardrail.ToolOrigin.RemoteMcp => new ToolCallGuardrail.ToolOrigin.RemoteMcp(mcp.endpoint)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a risk that we add another origin later? could be good to make it future proof like in #1997

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unlikely, but we can always protect it

octonato and others added 6 commits October 12, 2026 21:55
All use-for values are deprecated, for SimilarityGuard too. Remove the tests
of the separate SimilarityGuard warning. List the replacements for the MCP
values in the migration guide.
@octonato
octonato force-pushed the octonato/new-guardrails-for-mcp-sdk branch from 49558b0 to cd000cd Compare October 12, 2026 20:03
@patriknw
patriknw marked this pull request as ready for review October 13, 2026 08:41
@patriknw

Copy link
Copy Markdown
Contributor

🤖 review, judge if the suggestions are valid

  1. [should-fix] akka-javasdk/src/main/scala/akka/javasdk/impl/agent/GuardrailProvider.scala:222: an origin that the SDK does not know fails every tool-call guardrail.
    toToolOrigin matches the two cases with no fallback. It runs when the context is built, so it fails inside decide for every guardrail, also for guardrails that never read origin(). The runtime treats this as Fail. A blocking guardrail then stops every call of that kind with the reason scala.MatchError: .... A report-only guardrail logs a warning and publishes an error on every call. A scratch test with an origin the SDK does not know got Failure(scala.MatchError: null). Because of this, a later runtime cannot add a third place where a tool runs while SDK 3.7.0 is supported. The ledger client on main reads a value that it does not know as UNSPECIFIED (knownOr in LedgerClientImpl). This mapping needs the same treatment, whatever public shape is chosen. This is the open question on the PR.

  2. [should-fix] akka-javasdk/src/main/java/akka/javasdk/agent/ToolCallGuardrail.java:39: RemoteMcp prints credentials.
    The record toString gives RemoteMcp[endpoint=https://user:secret@mcp.example.com/mcp?api_key=k123] (verified). The interface type prints only RemoteMcp. In its own messages and in the decision record, the runtime strips the user info, the query and the fragment from the endpoint. Suppose a guardrail logs ctx.origin() or puts it in a deny reason. The runtime copies the reason into the exception message, the ledger decision record, the span and the GuardrailException that the agent receives. The javadoc of endpoint does not say that the value can hold credentials. Override toString and document the risk. Whether endpoint() should return the stripped form, which matches the decision record.

  3. [should-fix] akka-javasdk/src/main/scala/akka/javasdk/impl/agent/AgentImpl.scala:459, :486, akka-javasdk/src/main/java/akka/javasdk/agent/RemoteMcpTools.java:101: the change alters interceptor behaviour without saying so.
    With no guardrail involved, a service with an interceptor now sees these differences (verified on main and on the branch, see the table below):

    • For an MCP tool with an alias, both hooks get the name on the MCP server (echo). Before, they got the alias (echo_2). An interceptor that filters or logs by the alias changes behaviour.
    • The tool call intent record in the ledger holds the arguments that interceptRequest returns, not the arguments from the model. Code that joins it with the ToolCall of the interaction record from LedgerClient (model arguments) no longer matches when the hook rewrites them.
    • A null from interceptResponse now fails the call with McpToolCallExecutionException. In an autonomous agent it becomes an error result. On main the request ended with a generic component client error and onFailure was not called. This is an improvement.

    The first two follow from the two-hook contract of the runtime, and they are reasonable. But neither is stated. ToolInterceptorContext.toolName() still says "The tool name that the call is for". extending.adoc and the migration guide say nothing about it. fix: keep MCP tool error flag in interceptors and session memory #2041 states both in its description.

  4. [should-fix] No test pins the interceptor mapping, and the integration tests cover one route.
    Every interceptor user goes through the new McpToolCallHooksAdapter. I mutated it, and each of these mutations passed all 41 tests of AgentIntegrationTest and all agent unit tests:

    • the hooks get a wrong tool name
    • interceptResponse gets the model arguments instead of the arguments that were sent
    • the error flag is dropped

    These mutations were caught: an empty lookup, the one-function interceptor form, a request hook that is not called, and a response hook that is not called. The new integration tests cover a request-based agent with an interceptor, for an allow and for a blocking deny. Nothing covers:

    • an autonomous agent (SdkRunner.scala:1212 builds its descriptor separately, so the SDK can get that route wrong on its own)
    • an MCP server without an interceptor
    • a report-only deny or a guardrail that throws
    • a streamed reply
    • an old guardrail on an MCP call, alone or next to a tool-call guardrail

    Scratch tests show that all of these work today. At minimum, add the autonomous route, the case without an interceptor, and a unit test of the hook mapping. fix: keep MCP tool error flag in interceptors and session memory #2041 has such a spec (see the merge note).

  5. [should-fix] GuardrailProvider.scala:553-568: the similarity guard advice can drop a check, and it does not count a double check.
    The agent-level warning now always says "Remove the agents and agent roles from [x]". Take an old SimilarityGuard on [model-request, mcp-tool-request] next to "default model-call jailbreak" (output captured):
    Agent [a] has both the deprecated SimilarityGuard [old] with use-for [model-request] and the ModelCallSimilarityGuard [default model-call jailbreak]. Both check the user message and the tool results. Remove the agents and agent roles from [old].
    Following it removes the only similarity check on MCP arguments, which has no built-in replacement. main said "Set use-for on [old] to [mcp-tool-request]".
    The change also removes the note "count the MCP use-for values once an MCP replacement for SimilarityGuard exists" but does not do what the note says. mcp-tool-response now has a replacement, yet only model-request is counted. An agent with the old guard on mcp-tool-response and the new guard gets no warning, and each MCP result is searched twice. The old guard searches it raw, and the new guard searches it masked before the next model call. When the old entry also has model-request, the result is searched three times.
    Count model-request and mcp-tool-response. When the entry has other uses, tell the user to remove those two from use-for instead of removing the agents.

  6. [should-fix] docs/src/modules/reference/pages/migration-guide.adoc:15: the guide does not say what changes after the move.
    The runtime shows these differences when a service moves from an old guardrail on MCP results to a ModelCallGuardrail. The guide states none of them:

    • The guardrail sees the result after the tool-result sanitizers, not the raw result.
    • It runs before the next model call, after all calls of the round, not right after the MCP call.
    • A blocked result is already in the interaction record of the ledger. The old guardrail failed the call before that.
    • In an autonomous agent, some rounds end the iteration: a control tool call in the same response, a failed tool call, or the tool call limit. Such a round goes into session memory and the ledger with no model-call check in that iteration.
    • The failure names the model call ("Before model call guardrail blocked ..."), not the MCP response.

    For mcp-tool-request, the guide does not say that tool-call guardrails run before the old request guardrails on the same server. The old ones run only when no blocking guardrail stopped the call. The guide also does not say that the decision is now recorded in the ledger. A short list is enough.

  7. [nit] GuardrailProvider.scala:443: for "default jailbreak" with use-for = ["*"], the warning gives two replacements for one check:
    ... Instead, enable "default model-call jailbreak" for the same agents and agent roles, remove model-response, ..., use akka.javasdk.agent.ModelCallSimilarityGuard for mcp-tool-response.
    The default model-call entry already checks tool results. A user who follows both instructions adds a second ModelCallSimilarityGuard.

  8. [nit] migration-guide.adoc:17: the change for existing ToolCallGuardrail entries is under "Migrating from TextGuardrail".
    A 3.7.0-M1 user with a ToolCallGuardrail and no TextGuardrail skips that heading. Such a user can have a guardrail that denies unknown tool names, or one that parses the arguments by the schema of a function tool. When it is blocking, it now stops every MCP call. When it is report-only, it records a deny or a failure on every MCP call. Give this its own heading, or a release note. The insertion also moved "Remove use-for from the entry" (line 23) after the SimilarityGuard pointer, where it reads as part of that pointer.

  9. [nit] Leftovers.

    • AgentImpl.scala:445 says "see FIXME below", but the change removed that FIXME.
    • AgentImpl.scala:697 adds a second FIXME about the error flag in session memory. That is not part of this task. It also merges silently next to the fix in fix: keep MCP tool error flag in interceptors and session memory #2041, where it becomes untrue.
    • docs/src/modules/reference/pages/release-notes.adoc:20 still says that guardrails run "before a function tool call".
  10. [nit] The new warning text has some gaps.

    • "uses deprecated use-for values" does not name the values, or *, as the user wrote them.
    • For a guardrail of the user's own, ModelCallGuardrail appears twice, once for model-request and once for mcp-tool-response. The text does not say that one class covers both.
    • The default-entry advice drops the step "Then remove the agents and agent roles". "Instead" implies it, and the migration guide states it.
  11. [nit] docs/src/modules/sdk/pages/agents/guardrails.adoc, callout 8: tools and toolName() use the name on the MCP server, but the ledger records the name that the model used. That applies to the ToolCall of the interaction record and to the decision record. For an aliased tool, the two names differ. Add one line. A developer finds the matching entry through the guardrail name in the decision record, so this is only a naming note.

@patriknw

Copy link
Copy Markdown
Contributor

🤖 Note about merge with #2041

It also replaces the tool interceptor with McpToolCallHooks, in the same lines of AgentImpl. The two mappings behave the same. #2041 also gives the hooks isError() and keeps the flag in session memory, so its mapping covers everything this one does.

If #2041 merges first and this PR rebases on it:

  • Keep toSpiMcpToolCallHooks from #2041 instead of McpToolCallHooksAdapter. Keep toolCallGuardrails = guardrails.toolCallGuardrails from this PR.
  • Remove the FIXME at AgentImpl.scala:696. It merges in without a conflict and is untrue after #2041.
  • Keep one of AgentImplMcpToolsSpec and AgentImplMcpToolInterceptorSpec: the fromService case from this PR and the isError() cases from #2041.
  • Keep both new sentences in extending.adoc.

Both PRs change the tool name the hooks get for an aliased tool, and the arguments in the intent record. Document that in whichever PR lands first.

@patriknw patriknw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@octonato

Copy link
Copy Markdown
Contributor Author

@patriknw, I applied some more changes from your last reviews.

@octonato

Copy link
Copy Markdown
Contributor Author

Ah, I see you approved already. Fine, I will merge when green.

@octonato
octonato merged commit c86db45 into main Oct 13, 2026
38 checks passed
@octonato
octonato deleted the octonato/new-guardrails-for-mcp-sdk branch October 13, 2026 10:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation documentation related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants