Skip to content

fix: keep waiting in TaskClient.result() when a rule rejects a result - #2012

Merged
patriknw merged 5 commits into
mainfrom
fix/task-result-rejected-not-terminal
Oct 16, 2026
Merged

patriknw merged 5 commits into
mainfrom
fix/task-result-rejected-not-terminal

Conversation

@JustinPihony

@JustinPihony JustinPihony commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • TaskClient.result() and resultAsync() ignore ResultRejected notifications and keep waiting, since a rejected result is not terminal. The ignored rejection is logged at debug level.
  • Fix the TaskNotification javadoc, which said notifications are published only at terminal states, and document the behavior on TaskClient.result.

Fixes #2011

Test plan

  • shouldAwaitResultAcrossRuleRejection fails without the fix with Unexpected notification type ... ResultRejected and passes with it.
  • shouldFailResultAfterRepeatedRuleRejections: result() still throws TaskException.Failed when the iteration limit is reached.
  • Unit tests in TaskClientImplSpec cover rejected then completed, rejected then failed, and an already rejected task.
  • TaskRuleIntegrationTest passes.
  • CI checks run locally: javafmtCheckAll, headerCheckAll, scalafmtCheckAll, scalafix OrganizeImports, additionalValidation.

@JustinPihony

Copy link
Copy Markdown
Contributor Author

History check: resultAsync (#1457) predates ResultRejected. Before #1525 a rule rejection failed the task, so result() threw TaskException.Failed. #1525 made rejection non-terminal and added TaskException.ResultRejected, but did not update resultAsync, and nothing throws that exception. The Unexpected notification type branch was not a deliberate choice for rejections.

Ignoring the rejection does not hide a failure: the task still ends in FAILED after the iteration limit, and result() then throws TaskException.Failed. A rejection is also visible in the notification stream. This matches the documented contract that result() waits for COMPLETED, FAILED or CANCELLED. I added a debug log so an ignored rejection leaves a trace.

Not changed here: the final Failed reason does not include the last rule rejection.

@JustinPihony
JustinPihony marked this pull request as ready for review October 8, 2026 04:32
@patriknw

patriknw commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

I have confirmed that this is a real issue. The solution looks good.


🤖 review

Review: fix/task-result-rejected-not-terminal

  1. Update notifications.adoc (lines 148-150). It still says the task entity publishes "on every terminal transition" and that non-terminal transitions publish nothing. That contradicts the corrected TaskNotification javadoc. State that the entity also publishes ResultRejected, that it is not terminal, and that the agent retries. The ResultRejected table row can say the same.

  2. Bound the wait in shouldFailResultAfterRepeatedRuleRejections. It calls result() with no timeout, so a regression that makes the wait hang would hang the suite instead of failing the test. Use resultAsync(...).toCompletableFuture().get(15, TimeUnit.SECONDS) and assert on the cause. A JUnit @Timeout in its default mode does not help, because it interrupts the thread and result() blocks in CompletableFuture.join(), which ignores interrupts.

  3. Add unit tests to TaskClientImplSpec. The integration tests depend on timing to catch a regression, so the spec is the place for deterministic coverage.

    • Register classOf[TaskNotification.ResultRejected] in the spec serializer. It only registers the three terminal types today.
    • ResultRejected then Completed returns the result.
    • ResultRejected then Failed fails with TaskException.Failed.
    • A task already RESULT_REJECTED at subscribe time returns the result on the next Completed.
    • The existing mockEntityClient(state, notificationSource) helper accepts a Source with several notifications.

Suggestions

  1. Replace the 1 second sleep in shouldAwaitResultAcrossRuleRejection with a gate. The Awaitility.await().pollDelay(...) call is a sleep. If subscribing ever takes longer than a second, the test still passes but stops covering the fix. A gate ties the order to the call instead of to time, and removes the supplyAsync wrapper. This sketch is not run:

    var resultRequested = new CompletableFuture<Void>().orTimeout(10, TimeUnit.SECONDS);
    
    agentModel
        .whenMessage(msg -> msg.contains("Do something"))
        .reply(
            input -> {
              resultRequested.join(); // released after the test calls resultAsync
              return new TestModelProvider.AiResponse(
                  completeTask(new TestTasks.TestResult("low quality", 3)));
            });
    
    // second reply and runSingleTask as before
    
    var result = componentClient.forTask(taskId).resultAsync(TestTasks.VALIDATED_TASK);
    resultRequested.complete(null);
    
    assertThat(result.toCompletableFuture().get(15, TimeUnit.SECONDS).value())
        .isEqualTo("improved result");
  2. Consider ignoring every non-terminal notification, not only ResultRejected (TaskClientImpl.scala:151). The default branch still throws, so the next non-terminal notification type repeats this bug. The runtime's consumer of the same stream ignores anything that is not terminal (TaskProcessing.handleDependencyNotification). The trade-off is that a future terminal type that nobody handles would hang instead of fail.

Nit

  1. .map { notification => notification match { ... } } can be .map { case ... } (TaskClientImpl.scala:140).

@JustinPihony

Copy link
Copy Markdown
Contributor Author

Addressed in 7cc5d78:

  1. notifications.adoc now says the entity also publishes ResultRejected when a rule rejects a result, that it is not terminal, and that resultAsync keeps waiting. The table row says the same.
  2. The failing-path test waits with resultAsync(...).get(15, SECONDS) and asserts the TaskException.Failed cause.
  3. Added three cases to TaskClientImplSpec: rejected then completed, rejected then failed, and a task already RESULT_REJECTED at subscribe. The first two fail without the fix. The third passes either way, because an already rejected task has no rejection notification to ignore, so it only pins that the wait continues.
  4. The sleep is replaced with a gate that is released after resultAsync is called. Both integration tests fail without the fix.
  5. Not changed. Ignoring only ResultRejected keeps an unknown notification type loud: a future non-terminal type fails with the existing IllegalStateException instead of making result() wait until the caller gives up. I can switch to ignoring everything non-terminal, like handleDependencyNotification, if you prefer.
  6. Done.

@github-actions github-actions Bot added the documentation documentation related label Oct 8, 2026
@JustinPihony
JustinPihony requested a review from patriknw October 8, 2026 21:14

@patriknw patriknw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@patriknw
patriknw merged commit a4b05e6 into main Oct 16, 2026
38 checks passed
@patriknw
patriknw deleted the fix/task-result-rejected-not-terminal branch October 16, 2026 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation documentation related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TaskClient.result() throws on a rejected task result instead of waiting

2 participants