Repository navigation
Fix: issue 3231 retry infinite loop - #3251
Conversation
β¦false During test retries, if cacheDataForTestRetries was set to false, the DataProvider was re-executed but the new parameters were ignored. This change ensures that the new parameters are correctly used.
This test confirms that when cacheDataForTestRetries=false, the test method receives the fresh data generated by the DataProvider, rather than cached stale data.
Split tests into runners extending SimpleBaseTest and sample test cases, ensuring proper isolation and style consistency.
β¦oker.java Co-authored-by: Julien Herr <julien@herr.fr>
β¦/TestInvoker.java" This reverts commit 1b37b67.
Bumps the Gradle wrapper validation GitHub Action from v3 to v5 for improved security and compatibility.
β¦ue 3231) - Add getParameterIndex() to ITestResult interface. - Use parameterIndex in BaseTestMethod to identify test instances for RetryAnalyzer. - This ensures a stable key even if parameters are mutated. - Also fixes shared retry limit for rows with equal parameters. - Add regression tests Issue3231Test and RetryLimitTest.
β¦try-infinite-loop
|
Warning Rate limit exceeded@baflQA has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 16 minutes and 5 seconds before requesting another review. β How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. π¦ How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. π Files selected for processing (1)
π WalkthroughWalkthroughReplaces object-identity caching for DataProvider parameters with index-based tracking: adds ITestResult.getParameterIndex(), removes IdentifiableArrayObject and per-instance parameter cache, updates BaseTestMethod/TestResult/TestInvoker to use parameter index, adds retry-related tests/samples, and updates CHANGES.txt. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant DP as DataProvider
participant Runner as TestNG Runner
participant Base as BaseTestMethod
participant TR as TestResult
participant Test as Test Method
participant Retry as RetryAnalyzer
Note over DP,Runner: initial invocation (row N)
Runner->>DP: obtain parameters (row N)
DP-->>Runner: parameter object (row N)
Runner->>Base: invoke test with parameters
Base->>TR: create via TestResult.newTestResult(params, N)
TR-->>Base: stores parameter index N
Base->>Test: execute
Test-->>Runner: failure
Runner->>Retry: shouldRetry(TR)
Retry-->>Runner: decision (reads TR.getParameterIndex())
alt retry allowed
Runner->>Base: schedule retry with same parameter index N
else no retry
Runner->>TR: mark final failure
end
Estimated code review effortπ― 3 (Moderate) | β±οΈ ~20 minutes Possibly related PRs
Suggested reviewers
Poem
π₯ Pre-merge checks | β 4 | β 1β Failed checks (1 warning)
β Passed checks (4 passed)
βοΈ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
β¦om/baflQA/testng into fix/issue-3231-retry-infinite-loop
β¦om/baflQA/testng into fix/issue-3231-retry-infinite-loop
|
@coderabbitai generate docstrings |
β Actions performedInitiated docstring generation; will generate only if new commits exist. |
|
Note Docstrings generation - SUCCESS |
Docstrings generation was requested by @baflQA. * #3251 (comment) The following files were modified: * `testng-core-api/src/main/java/org/testng/ITestResult.java` * `testng-core/src/main/java/org/testng/internal/BaseTestMethod.java` * `testng-core/src/test/java/test/retryAnalyzer/issue3231/samples/MutationSample.java` * `testng-core/src/test/java/test/retryAnalyzer/issue3231/samples/RetryLimitSample.java` * `testng-runner-api/src/main/java/org/testng/internal/TestResult.java`
| IObject.IdentifiableArrayObject parameter = | ||
| new IObject.IdentifiableArrayObject(itr.getParameters()); | ||
| return parameters.computeIfAbsent(parameter, Function.identity()).getInstanceId(); | ||
| return Integer.toString(itr.getParameterIndex()); |
There was a problem hiding this comment.
@krmahadevan I let you review this part as you made most of it if I remember well.
There was a problem hiding this comment.
Results:
- Command: ./gradlew :testng-core:test --tests test.retryAnalyzer.RetryAnalyzerTest.ensureRetryDoesntRunEndlesslyForDataDrivenTests -Ptestng.test.extra.jvmargs="-Duser.timezone=America/New_York -Duser.country=RU -Duser.language=ru
-XX:+UnlockExperimentalVMOptions -XX:hashCode=2" - Outcome: BUILD SUCCESSFUL (1/1 tests passed)
All good i think?
|
Hey @krmahadevan, can i count on Your review? |
| IObject.IdentifiableArrayObject parameter = | ||
| new IObject.IdentifiableArrayObject(itr.getParameters()); | ||
| return parameters.computeIfAbsent(parameter, Function.identity()).getInstanceId(); | ||
| return Integer.toString(itr.getParameterIndex()); |
| String id(); | ||
|
|
||
| /** @return - The index of the parameter row when a @DataProvider is being used. */ | ||
| default int getParameterIndex() { |
There was a problem hiding this comment.
This logic relies on the parameter index which is mutable. So that can cause the internal state to change and behavior become unpredictable. I think we would need to ensure that we dont allow that.
There was a problem hiding this comment.
Any specific suggestion?
There was a problem hiding this comment.
I've made some proposal
There was a problem hiding this comment.
Actionable comments posted: 0
π§Ή Nitpick comments (3)
testng-core/src/test/java/test/retryAnalyzer/RetryAnalyzerTest.java (1)
51-86: LGTM! Well-structured regression tests for issue #3231.Both test methods correctly verify the fix:
testMutationDoesNotCauseInfiniteRetryLoopvalidates that mutating data provider objects no longer causes infinite retries.testEachRowHasItsOwnRetryLimitconfirms that data provider rows with identical parameter values maintain independent retry counters.The use of
IInvokedMethodListenerto count actual invocations is a solid approach, and the expected counts (4 in both cases) align with the retry analyzer configurations.Optional: Extract common listener pattern
The two test methods share nearly identical listener setup. Consider extracting a helper method to reduce duplication:
private int runTestAndCountInvocations(Class<?> testClass) { AtomicInteger invCount = new AtomicInteger(0); TestNG tng = create(testClass); tng.addListener( new IInvokedMethodListener() { @Override public void afterInvocation(IInvokedMethod method, ITestResult testResult) { if (method.isTestMethod()) { invCount.incrementAndGet(); } } }); tng.run(); return invCount.get(); }Then simplify the tests:
@Test(description = "GITHUB-3231") public void testMutationDoesNotCauseInfiniteRetryLoop() { MutationSample.guardCounter.set(0); assertThat(runTestAndCountInvocations(MutationSample.class)).isEqualTo(4); } @Test(description = "GITHUB-3231") public void testEachRowHasItsOwnRetryLimit() { assertThat(runTestAndCountInvocations(RetryLimitSample.class)).isEqualTo(4); }testng-core/src/test/java/test/retryAnalyzer/issue3231/RetryLimitSample.java (1)
11-19: LGTM! Retry logic is correct.The
MyRetryimplementation allows exactly 1 retry (2 total attempts per data provider row), which correctly tests that identical parameter values receive independent retry counters.Note: This implementation uses increment-then-check style (
count++; return count < 2), whileMutationSample.MyRetry(lines 49-51 in MutationSample.java) uses check-then-increment style. Both are correct but stylistically inconsistent. Consider standardizing if adding more retry analyzers in the future.Based on learnings, hardcoding retry limits in test code is acceptable for the TestNG project.
testng-core/src/test/java/test/retryAnalyzer/issue3231/MutationSample.java (1)
64-71: LGTM! Test correctly validates the mutation scenario.The guard counter (line 66) is a defensive safety mechanism to prevent infinite loops during test development. The mutation on line 69 reproduces the original issue where changing parameter object identity would reset retry counters.
Optional: Add explicit guardCounter assertion
Consider asserting the guard counter value in
RetryAnalyzerTest.testMutationDoesNotCauseInfiniteRetryLoop()to make the test more explicit:@Test(description = "GITHUB-3231") public void testMutationDoesNotCauseInfiniteRetryLoop() { MutationSample.guardCounter.set(0); AtomicInteger invCount = new AtomicInteger(0); // ... listener setup and run ... assertThat(invCount.get()).isEqualTo(4); assertThat(MutationSample.guardCounter.get()).isEqualTo(4); // Guard never triggered }This would explicitly verify that the fix prevents infinite loops rather than relying on the guard's implicit timeout.
π Review details
Configuration used: defaults
Review profile: CHILL
Plan: Pro
π Files selected for processing (3)
testng-core/src/test/java/test/retryAnalyzer/RetryAnalyzerTest.javatestng-core/src/test/java/test/retryAnalyzer/issue3231/MutationSample.javatestng-core/src/test/java/test/retryAnalyzer/issue3231/RetryLimitSample.java
π§° Additional context used
π§ Learnings (3)
π Common learnings
Learnt from: krmahadevan
Repo: testng-team/testng PR: 3181
File: testng-core/src/test/java/test/invocationcount/issue3180/RetryAnalyzer.java:6-9
Timestamp: 2024-10-09T09:04:04.750Z
Learning: In test code for the TestNG project, it's acceptable to hardcode parameters like `retryLimit` since they don't need to be configurable.
Learnt from: krmahadevan
Repo: testng-team/testng PR: 3181
File: testng-core/src/test/java/test/invocationcount/issue3180/RetryAnalyzer.java:6-9
Timestamp: 2024-09-30T05:21:21.304Z
Learning: In test code for the TestNG project, it's acceptable to hardcode parameters like `retryLimit` since they don't need to be configurable.
Learnt from: krmahadevan
Repo: testng-team/testng PR: 3181
File: testng-core/src/test/java/test/invocationcount/issue3180/RetryAnalyzer.java:11-16
Timestamp: 2024-10-09T09:04:04.750Z
Learning: The `retry(ITestResult result)` method must retain the `ITestResult` parameter because it implements the `IRetryAnalyzer` interface, and method signatures cannot be altered when implementing interfaces, even if the parameter is unused.
Learnt from: krmahadevan
Repo: testng-team/testng PR: 3181
File: testng-core/src/test/java/test/invocationcount/issue3180/RetryAnalyzer.java:11-16
Timestamp: 2024-09-30T05:21:00.298Z
Learning: The `retry(ITestResult result)` method must retain the `ITestResult` parameter because it implements the `IRetryAnalyzer` interface, and method signatures cannot be altered when implementing interfaces, even if the parameter is unused.
π Learning: 2024-09-30T05:21:21.304Z
Learnt from: krmahadevan
Repo: testng-team/testng PR: 3181
File: testng-core/src/test/java/test/invocationcount/issue3180/RetryAnalyzer.java:6-9
Timestamp: 2024-09-30T05:21:21.304Z
Learning: In test code for the TestNG project, it's acceptable to hardcode parameters like `retryLimit` since they don't need to be configurable.
Applied to files:
testng-core/src/test/java/test/retryAnalyzer/RetryAnalyzerTest.javatestng-core/src/test/java/test/retryAnalyzer/issue3231/RetryLimitSample.javatestng-core/src/test/java/test/retryAnalyzer/issue3231/MutationSample.java
π Learning: 2024-09-30T05:21:00.298Z
Learnt from: krmahadevan
Repo: testng-team/testng PR: 3181
File: testng-core/src/test/java/test/invocationcount/issue3180/RetryAnalyzer.java:11-16
Timestamp: 2024-09-30T05:21:00.298Z
Learning: The `retry(ITestResult result)` method must retain the `ITestResult` parameter because it implements the `IRetryAnalyzer` interface, and method signatures cannot be altered when implementing interfaces, even if the parameter is unused.
Applied to files:
testng-core/src/test/java/test/retryAnalyzer/issue3231/RetryLimitSample.javatestng-core/src/test/java/test/retryAnalyzer/issue3231/MutationSample.java
𧬠Code graph analysis (2)
testng-core/src/test/java/test/retryAnalyzer/issue3231/RetryLimitSample.java (1)
testng-core/src/test/java/test/retryAnalyzer/issue3231/MutationSample.java (1)
MyRetry(43-55)
testng-core/src/test/java/test/retryAnalyzer/issue3231/MutationSample.java (2)
testng-core/src/test/kotlin/org/testng/internal/samples/classhelper/issue1339/BabyPandaSample.kt (2)
equals(8-17)hashCode(19-19)testng-core/src/test/java/test/retryAnalyzer/issue3231/RetryLimitSample.java (1)
MyRetry(11-19)
β° Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: 17, temurin, ubuntu, America/New_York, tr_TR, stress JIT
- GitHub Check: 11, temurin, windows, UTC, tr_TR, stress JIT
- GitHub Check: 11, corretto, macos, Pacific/Chatham, ru_RU, stress JIT
- GitHub Check: 26, oracle, macos, UTC, ru_RU
- GitHub Check: 21, liberica, ubuntu, UTC, ru_RU, stress JIT
- GitHub Check: 25, zulu, same hashcode, windows, America/New_York, fr_FR
- GitHub Check: 26, oracle, macos, Pacific/Chatham, ru_RU, stress JIT
π Additional comments (4)
testng-core/src/test/java/test/retryAnalyzer/issue3231/RetryLimitSample.java (2)
21-24: LGTM! Data provider correctly tests identical parameters.Returning two rows with identical values
"a"is intentional and effectively tests that parameter value equality doesn't cause retry analyzer instances to be shared between different data provider rowsβthe core issue this PR addresses.
26-29: LGTM! Test method is appropriately simple.The test correctly fails unconditionally to trigger the retry analyzer and validate the fix.
testng-core/src/test/java/test/retryAnalyzer/issue3231/MutationSample.java (2)
14-41: LGTM! Standard mutable object implementation.The
Customclass correctly implementsequals()andhashCode()based on the mutableidfield usingObjectsutilities. This implementation is key to reproducing the original issueβmutating theidchanges the object's identity, which previously caused retry counters to reset.
43-55: LGTM! Retry analyzer correctly configured.The
MyRetryimplementation allows 3 retries (4 total attempts), which aligns with the test expectation inRetryAnalyzerTest.testMutationDoesNotCauseInfiniteRetryLoop()(line 66).Based on learnings, hardcoding retry limits in test code is acceptable for the TestNG project.
There was a problem hiding this comment.
Actionable comments posted: 0
π§Ή Nitpick comments (3)
testng-core/src/test/java/test/retryAnalyzer/RetryAnalyzerTest.java (3)
52-59: Consider adding a test description for consistency.The test logic is correct (2 invocations Γ 2 attempts each = 4 total). However, the other new tests (lines 61, 79) include a
descriptionattribute. Consider adding one here for consistency, such as@Test(description = "GITHUB-3231").
61-77: LGTM! The mutation test correctly prevents infinite retry loops.The test properly verifies that parameter mutation doesn't cause infinite retries. The expected count (4 = 1 original + 3 retries) is correct based on MutationSample's retry configuration.
Optional: Verify the guardCounter safety mechanism
Consider adding an assertion to confirm the guardCounter didn't hit its safety limit:
tng.run(); assertThat(invCount.get()).isEqualTo(4); + assertThat(MutationSample.guardCounter.get()).isEqualTo(4);This would verify that the safety mechanism (which returns early if counter > 100) was never triggered, providing additional confidence that the fix works correctly.
79-96: LGTM! This test correctly verifies independent retry limits per DataProvider row.The test validates a critical aspect of the fix: distinct DataProvider rows with identical parameter values receive independent RetryAnalyzer instances. The expected count (4 = 2 rows Γ 2 attempts each) is correct.
Optional: Reduce listener duplication
The listener setup at lines 83-91 is nearly identical to lines 66-74. If desired, you could extract a helper method:
private IInvokedMethodListener createInvocationCounter(AtomicInteger counter) { return new IInvokedMethodListener() { @Override public void afterInvocation(IInvokedMethod method, ITestResult testResult) { if (method.isTestMethod()) { counter.incrementAndGet(); } } }; }Then use:
tng.addListener(createInvocationCounter(invCount));However, the current approach is clear and acceptable in test code.
π Review details
Configuration used: defaults
Review profile: CHILL
Plan: Pro
π Files selected for processing (3)
testng-core/src/test/java/test/retryAnalyzer/RetryAnalyzerTest.javatestng-core/src/test/java/test/retryAnalyzer/issue3231/InvocationCountRetrySample.javatestng-runner-api/src/main/java/org/testng/internal/TestResult.java
π§ Files skipped from review as they are similar to previous changes (1)
- testng-core/src/test/java/test/retryAnalyzer/issue3231/InvocationCountRetrySample.java
π§° Additional context used
π§ Learnings (2)
π Learning: 2024-09-30T05:21:21.304Z
Learnt from: krmahadevan
Repo: testng-team/testng PR: 3181
File: testng-core/src/test/java/test/invocationcount/issue3180/RetryAnalyzer.java:6-9
Timestamp: 2024-09-30T05:21:21.304Z
Learning: In test code for the TestNG project, it's acceptable to hardcode parameters like `retryLimit` since they don't need to be configurable.
Applied to files:
testng-core/src/test/java/test/retryAnalyzer/RetryAnalyzerTest.javatestng-runner-api/src/main/java/org/testng/internal/TestResult.java
π Learning: 2024-10-09T09:04:04.750Z
Learnt from: krmahadevan
Repo: testng-team/testng PR: 3181
File: testng-core/src/test/java/test/invocationcount/issue3180/RetryAnalyzer.java:11-16
Timestamp: 2024-10-09T09:04:04.750Z
Learning: The `retry(ITestResult result)` method must retain the `ITestResult` parameter because it implements the `IRetryAnalyzer` interface, and method signatures cannot be altered when implementing interfaces, even if the parameter is unused.
Applied to files:
testng-core/src/test/java/test/retryAnalyzer/RetryAnalyzerTest.javatestng-runner-api/src/main/java/org/testng/internal/TestResult.java
𧬠Code graph analysis (1)
testng-core/src/test/java/test/retryAnalyzer/RetryAnalyzerTest.java (3)
testng-core/src/test/java/test/retryAnalyzer/issue3231/InvocationCountRetrySample.java (1)
InvocationCountRetrySample(9-27)testng-core/src/test/java/test/retryAnalyzer/issue3231/MutationSample.java (1)
MutationSample(12-72)testng-core/src/test/java/test/retryAnalyzer/issue3231/RetryLimitSample.java (1)
RetryLimitSample(9-30)
β° Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: 26, oracle, ubuntu, UTC, ru_RU
- GitHub Check: 25, zulu, same hashcode, macos, UTC, tr_TR, stress JIT
- GitHub Check: 17, zulu, ubuntu, America/New_York, fr_FR, stress JIT
- GitHub Check: 17, microsoft, windows, UTC, ru_RU, stress JIT
- GitHub Check: 21, oracle, windows, UTC, tr_TR
- GitHub Check: 21, oracle, windows, UTC, ru_RU, stress JIT
- GitHub Check: 11, microsoft, ubuntu, Pacific/Chatham, fr_FR
π Additional comments (6)
testng-runner-api/src/main/java/org/testng/internal/TestResult.java (4)
39-39: LGTM!The field initialization to
-1correctly aligns with the default return value specified inITestResult.getParameterIndex(), serving as a clear sentinel for "unset."
357-362: LGTM! Guard logic correctly prevents conflicting reassignments.The condition allows initial assignment and idempotent re-sets while throwing on conflicting values β this is the right protection for ensuring stable retry identification.
364-367: LGTM!The
@Overrideannotation ensures the interface contract is properly satisfied.
369-377: LGTM!Adding
@Overrideannotations to these existing interface implementations improves maintainability and ensures compile-time verification of the interface contracts.Also applies to: 379-380, 456-458
testng-core/src/test/java/test/retryAnalyzer/RetryAnalyzerTest.java (2)
11-11: LGTM! Imports are appropriate.The new imports support the invocation counting mechanism used in the tests below.
Also applies to: 15-16
46-48: LGTM! Sample class imports are correctly added.These imports reference the test fixtures that exercise the issue #3231 fixes.
|
@baflQA - Please check this patch. We could use something like this. We can also add a deprecation message to the |
|
hey @krmahadevan - I've applied Your suggestions, tests are still passing ;) I think that we are approaching the happy ending :) |
There was a problem hiding this comment.
Actionable comments posted: 1
π€ Fix all issues with AI agents
In @testng-runner-api/src/main/java/org/testng/internal/TestResult.java:
- Around line 367-368: Mark the no-op setter setParameterIndex(int
parameterIndex) as deprecated: add the @Deprecated annotation to the method and
update its Javadoc with a @deprecated tag explaining that the method is a no-op
and will be removed in a future release (optionally mention the recommended
alternative or that callers should stop using it). Ensure the annotation and
Javadoc appear on the method signature for TestResult.setParameterIndex so tools
and IDEs show the deprecation to users.
π Review details
Configuration used: defaults
Review profile: CHILL
Plan: Pro
π Files selected for processing (4)
testng-core-api/src/main/java/org/testng/ITestResult.javatestng-core/src/main/java/org/testng/internal/IObject.javatestng-core/src/main/java/org/testng/internal/invokers/TestInvoker.javatestng-runner-api/src/main/java/org/testng/internal/TestResult.java
π€ Files with no reviewable changes (1)
- testng-core/src/main/java/org/testng/internal/IObject.java
π§ Files skipped from review as they are similar to previous changes (1)
- testng-core-api/src/main/java/org/testng/ITestResult.java
π§° Additional context used
π§ Learnings (2)
π Learning: 2024-10-09T09:04:04.750Z
Learnt from: krmahadevan
Repo: testng-team/testng PR: 3181
File: testng-core/src/test/java/test/invocationcount/issue3180/RetryAnalyzer.java:11-16
Timestamp: 2024-10-09T09:04:04.750Z
Learning: The `retry(ITestResult result)` method must retain the `ITestResult` parameter because it implements the `IRetryAnalyzer` interface, and method signatures cannot be altered when implementing interfaces, even if the parameter is unused.
Applied to files:
testng-runner-api/src/main/java/org/testng/internal/TestResult.java
π Learning: 2024-10-09T09:04:04.750Z
Learnt from: krmahadevan
Repo: testng-team/testng PR: 3181
File: testng-core/src/test/java/test/invocationcount/issue3180/RetryAnalyzer.java:6-9
Timestamp: 2024-10-09T09:04:04.750Z
Learning: In test code for the TestNG project, it's acceptable to hardcode parameters like `retryLimit` since they don't need to be configurable.
Applied to files:
testng-runner-api/src/main/java/org/testng/internal/TestResult.java
𧬠Code graph analysis (1)
testng-core/src/main/java/org/testng/internal/invokers/TestInvoker.java (1)
testng-runner-api/src/main/java/org/testng/internal/TestResult.java (1)
TestResult(25-501)
β° Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: 26, oracle, macos, Pacific/Chatham, fr_FR, stress JIT
- GitHub Check: 21, temurin, windows, America/New_York, tr_TR
- GitHub Check: 21, zulu, ubuntu, UTC, de_DE, stress JIT
- GitHub Check: 11, liberica, windows, America/New_York, ru_RU
- GitHub Check: 25, corretto, windows, Pacific/Chatham, de_DE
- GitHub Check: 11, zulu, same hashcode, ubuntu, UTC, tr_TR, stress JIT
- GitHub Check: 17, zulu, ubuntu, America/New_York, de_DE
π Additional comments (6)
testng-runner-api/src/main/java/org/testng/internal/TestResult.java (5)
52-57: LGTM: Clean factory method for parameter-indexed results.The factory method correctly initializes both parameters and parameter index in a single call, providing a cleaner API than separate setter calls.
88-89: LGTM: Consistent use of factory method.Correctly uses the new factory to preserve both parameters and parameter index when creating a derived TestResult.
363-365: LGTM: Good encapsulation with private initialization.The private visibility ensures parameter index can only be set through the factory method, preventing external modification.
369-382: LGTM: Proper @OverRide annotations.The @OverRide annotations correctly indicate these methods implement the ITestResult interface.
Also applies to: 461-464
384-446: LGTM: @OverRide annotation added to interface implementation.The @OverRide annotation correctly indicates this method implements ITestResult.getSkipCausedBy().
testng-core/src/main/java/org/testng/internal/invokers/TestInvoker.java (1)
595-596: LGTM: Correct usage of new factory method.The change correctly uses
TestResult.newTestResult()to create a TestResult with both parameter values and parameter index in a single call, replacing the previous pattern of creating an empty result and setting fields via setters.
Corrected several typos in comments and variable names within TestResult.java, such as 'atleast' to 'at least', 'interection' to 'intersection', and 'allfailures' to 'allFailures'. Also updated deprecated annotation to specify forRemoval=true for setParameterIndex.
Removed an extraneous asterisk from the Javadoc comment in the isNotRunning() method for improved readability.
|
@baflQA This process wasnβt short or simple. Thank you for your patience and for the quality of your contributions throughout the discussion. Looking forward to seeing more pull requests from you in the future. |
|
Thanks for Your and @krmahadevan support on this matter. This issue was bothering me for too long π |
Following was done - Add getParameterIndex() to ITestResult interface. - Use parameterIndex in BaseTestMethod to identify test instances for RetryAnalyzer. - This ensures a stable key even if parameters are mutated. - Also fixes shared retry limit for rows with equal parameters. - Add regression tests Issue3231Test and RetryLimitTest. --------- Co-authored-by: Bartek Florczak <b.florczak@sap.com> Co-authored-by: Julien Herr <julien@herr.fr>
Fixes #3231.
Did you remember to?
CHANGES.txt./gradlew autostyleApplyDescription
This pull request fixes an issue where TestNG's
RetryAnalyzercould enter an infinite loop if a test method using aDataProvidermutated the parameter objects before failing.The Problem
Previously,
BaseTestMethodidentified test instances for retry analysis using a key generated from the parameter values'hashCode()andequals(). If a test mutated these parameters (e.g., changing an ID field used inhashCode), the next retry attempt would be seen as a new test instance. This caused TestNG to instantiate a freshRetryAnalyzerfor every attempt, resetting the retry counter and leading to an infinite loop. Additionally, differentDataProviderrows with identical parameter values would share the sameRetryAnalyzerinstance, causing them to incorrectly share a retry limit.The Fix
getParameterIndex()to theITestResultinterface. This index is uniquely and stably assigned by theMethodRunnerfor each row of aDataProvider.BaseTestMethod.parameterId()to useITestResult.getParameterIndex()instead of the parameter-basedIdentifiableArrayObject. This ensures the identification key remains constant for a specific test invocation, even if its parameters are mutated.RetryAnalyzerinstances and counters.Changes
testng-core-api: AddedgetParameterIndex()toITestResult.testng-runner-api: UpdatedTestResultto implement and override new methods correctly.testng-core:BaseTestMethodto useparameterIndexfor retry analyzer lookup.IdentifiableArrayObjectrelated logic fromBaseTestMethod.Issue3231Testto verify the fix for mutated parameters.RetryLimitTestto verify independent retry limits for identical parameters.samplespackage to prevent them from causing build failures in the main suite run.Summary by CodeRabbit
New Features
Bug Fixes
Tests
βοΈ Tip: You can customize this high-level summary in your review settings.