Skip to content

Deflake UI regression tests instead of quarantining them - #38822

Draft
kubaflo wants to merge 43 commits into
mainfrom
kubaflo-reliable-quarantined-tests
Draft

kubaflo wants to merge 43 commits into
mainfrom
kubaflo-reliable-quarantined-tests

Conversation

@kubaflo

@kubaflo kubaflo commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Note

Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!

Description of Change

Alternative to the test quarantines in #38777. This branch starts from main, does not apply that PR's quarantine patch, and keeps its 126 targeted method/platform pairs enabled wherever they were enabled on main.

Original quarantine scope Pairs
Android 26
iOS 14
MacCatalyst 73
Windows 13

Replace incidental screenshot checks with native geometry, visible content, ordering, navigation, and enabled-state assertions where those directly express the regression. Keep visual assertions and existing goldens where they are needed. Apple Shell search-icon correctness is instead covered by native device tests; Mac CollectionView selection retains rendered-color assertions without comparing an entire window.

The shared causes addressed here are:

  • Fullscreen state leaking across Mac fixtures. The original failing captures contained an actual fullscreen native window, not a screenshot-cropping error. Fullscreen tests restore the original native window size in unconditional cleanup, reacquire native windows across transitions, and move the pointer away from the green window control so its fullscreen/tiling hover menu cannot contaminate subsequent captures.
  • NUnit retaining failed polling assertions. Catching an assertion exception did not remove the failed assertion from the enclosing NUnit result. Polling attempts use isolated NUnit contexts, retain final failures, and retry only assertion/stale-element/missing-element failures. Four direct NUnit tests cover transient success, persistent failure, multiple assertions, and nontransient exceptions.
  • Unrelated Android tests loading Google Maps at startup. Original logcat evidence showed maps_dynamite module updates killing the host during CollectionView tests. The test host registers the real map handlers but initializes the Maps SDK only when a map is actually created; activity bundle handling and map-navigation coverage remain intact.
  • Explicit Android device selection being ignored. The runner set Udid in its configuration but did not forward it to Appium. It now passes the requested device capability, matching the iOS runner, rather than allowing another connected emulator and stale host to be selected.
  • Google Search opening external update UI over HostApp. Second-iteration Android Shell screenshots, native hierarchies and launching UIDs identify Google Search opening a Play Services/Play Store update before the tests' first interaction. Preparation disables only Google Search for user 0 on the disposable emulator created by a CI UI-test run. It verifies the serial and exact AVD name, checks package-manager output and independently reads back the state. Play Services, Play Store, Maps and Google API providers remain enabled; local/attached-device runs are not automatically changed.
  • Races and shared fixture state. Wait for actual native layout, scrolling, picker, keyboard, and navigation conditions; initialize independent scenarios explicitly; reacquire elements across rotation; avoid duplicate native identifiers and keyboard-corrupted sample text. Numeric fixture input uses invariant parsing. Follow-up fixes use native toolbar/tab identifiers, whitespace-free grouped-item IDs, text-aware scrolling, and a real detail page for native Back-command coverage.

Additional coverage includes eight native iOS/MacCatalyst Shell search-handler cases. WebView history exercises real navigation between bundled documents while retaining its initial HtmlWebViewSource, without depending on an external website. MacCatalyst supplies the actual bundle resource directory for relative navigation; iOS retains its default-base coverage.

The CI-driven correction batches also address these observed failures:

  • Windows Issue32223: retain the original W3C touch gesture, which raised ReorderCompleted in CI, but drop in the target cell's leading quarter instead of on its insertion boundary. W3C mouse is unsupported, and the intermediate native windows: clickAndDrag attempt was accepted without starting a reorder; neither is retained. Both the event and final item order remain required.
  • Android Issue32223: use the same once-only W3C touch sequence and drop one quarter of the target label's width left of its center. AndroidX ItemTouchHelper.chooseDropTarget requires strictly crossing the target's left edge for this leftward swap; equal-sized center-to-center cells land exactly on that boundary. The original actual NUnit test reproduces Waiting instead of Success on API30. The correction passes twice on API30 and once on API36, retaining both the completion event and David-before-Charlie geometry checks, with no gesture or whole-test retry. Windows and Apple behavior are unchanged by this follow-up.
  • Issue29588: use overlapping 0.8-viewport gestures on Windows and Mac, and bound traversal by thirty gestures (the fixture contains at most thirty items), not native-driver round-trip time. A later Mac run still exhausted the original fifteen-second budget with only Items10-17 visible; its successful retry needed four gestures and many native queries. Inspect the final permitted gesture and retain full native visibility, the actual threshold command, and newly loaded content as required assertions. Missing or permanently clipped targets still fail.
  • Mac Issue20062: verify native selection and rendered blue fill/red stroke/white normal state for every item, instead of a whole-window golden. Selecting Item0 and Item2 must affect both selected items while leaving the other two normal.
  • Apple Issue35736: iOS26's mismatch was solely the blinking insertion caret. All four original UI methods remain enabled as interaction/query smoke tests. Actual icon identity, dimensions, template mode, control states, cached clear-button replacement, and reset are checked by the native device cases; query text alone is not claimed as icon correctness.
  • Windows Shell RTL: WinAppDriver /location returns mirrored window coordinates, while its native accessibility source contains physical screen bounds. Assert visible, positive-sized, non-overlapping button and tab bounds from one native source snapshot before and after RTL, rather than using the mirrored coordinates.
  • Windows FlyoutPage title: start this scenario with fresh native controls rather than reusing controls transformed by the preceding RTL test. Keep its title actions, golden and tolerance unchanged; do not reset unrelated ordered scenarios that intentionally share navigation state.
  • iOS Issue9088: remove the inherited iOS26 runtime ignore after repeated simulator passes. All twelve swipe/count/closing-position checks remain.
  • Mac Shell transitions: reacquire native tabs when navigation replaces an element between lookup and its visibility query, using the existing bounded, isolated assertion polling. A stale reference never counts as proof that a tab is hidden. Locate the native back arrow inside the navigation bar by the previous page label; the failed CI hierarchy exposes Home, not a Back identifier, and also contains a separate Home tab.
  • Script validation gates: synchronize the trusted Retina patch context with the screenshot helper's protected visibility, without changing its patch operations. Correct two stale model-pin assertions to the existing GPT-6 Astra local-reviewer configuration; the configured model, no-override checks and long-context checks are unchanged.
  • Android ActivityStateManager regression fixtures: register callbacks, dispatch the synthetic event, assert, and dispose the test manager in one synchronous UI-thread turn. Real application lifecycle events cannot interleave with the exactly-one-event measurement, and both repeated-Init tests now unregister their callbacks. Keep listener-identity and event-count assertions unchanged.
  • Issue6994: dispatch the fixture's collection clear onto the UI thread rather than racing native ListView layout. Retain the intentional worker-thread label-color updates and Label removal/disposal stress from the original regression.
  • Issue2951: observe native button-count and complete text transitions between one-time clicks before inspecting the next item. CI8 exposed a distinct stale reference before the first click on Mac, so the follow-up gives A/B/C stable button identifiers and reacquires B/B/C immediately before each once-only tap, instead of holding a positional list of native elements. Retain the original counts, text/removal checks, background-color screenshot, golden and tolerance; neither actions nor whole tests are retried. The final version passes Android and iOS locally; CI10 subsequently confirms all five observed configurations, including the first Mac execution.
  • ScrollViewNoContent: wait for the existing Success label, which is set only after the empty ScrollView changes to non-null content and back to null. The old absence assertion returned prematurely or failed when the scenario had actually finished; the failed Mac hierarchy contains the visible Success label. The host transitions are unchanged.
  • Issue24996: measure each translation independently, excluding initial layout and updates to the diagnostic labels from its ancestor counters. Wait for a unique completed-step marker before requiring Lvl1[0/0], instead of accepting a stale zero or polling until the desired value appears. Preserve all four translations and the original measurement interval.
  • Issue26598 native regression coverage: isolate navigation from the hidden Home root to the visible InnerTab page, asserting native content, navigation-stack state, tab geometry and actual hit testing, with hidden/alpha/frame/window diagnostics. An additional Mac-only full routed case uses the production WindowHandler and its WindowViewController containment rather than the modal device-test stub. It verifies the hidden page, pop, Recent/Home selection and return to the hidden root, restoring the original native root and scene title before disconnecting test handlers. The original all-Apple first-transition case and end-to-end UI test remain enabled. The legacy UIKit root cause and product correction are described below.
  • Android Material3 Issue35119: wait for the prompt keyboard, send ESC once, and observe that it finishes hiding without closing the prompt. Appium's HideKeyboard can send a fallback BACK after a short keyboard-state wait, dismissing the dialog when the IME transition is slow. Require the prompt and Cancel to remain present and verify all three dialog responses. All three original styling screenshots, tolerances and comparison timeouts remain unchanged.
  • Windows Issue18896: the first-failure teardown hierarchy reports a 10-pixel residual ListView offset, despite the intended down/up round trip. Require native scroll progress, then use at most three upward gestures to reach an observed zero offset before the unchanged screenshot comparison. The first-failure comparison image itself was unavailable; the measured offset is from its preserved teardown hierarchy, not an inferred screenshot-cropping defect.
  • Windows Issue30536: move the physical mouse outside and then into the border using the supported windows: click extension, rather than relying on element invocation to raise pointer events. Observe the unchanged exact enter/exit counts (1/0), without repeating input, and close the secondary window in finally even when assertions fail. The initial correction used the shared W3C mouse helper; a follow-up normal commit replaced it because WinAppDriver supports pen/touch W3C actions, not mouse. The latest correction adds the live application-window origin to WinAppDriver's window-relative element bounds because windows: click takes screen-absolute coordinates. The host reports actual Created/Minimized state instead of treating a permanently present minimize button as readiness. No shared gesture implementation or downstream golden is changed.
  • GC assertion final verdict: extract the existing collector assertions into a platform-neutral partial, then check all references' current liveness after their existing waits. Previously, an early retained-reference verdict remained false even if a later collection released that object, producing a failure with no living-reference details. Eight source-linked unit cases cover deterministic late collection, strongly held targets in each array position, already-collected targets and invalid input. Collection order, budget and scheduling are unchanged; no JVM or bridge collection is added, and genuinely retained objects still fail.
  • Android recreation cleanup: finish any remaining current activity, dispose/unregister the test manager and reset fixture state together on the UI thread in all three recreation-test finally blocks. CI's original Mono crash identifies activation of a disposed lifecycle listener during OnActivityPrePaused. This does not establish the cause of the separate CoreCLR ART crash; both original failures remain recorded until verified on the corrected head.
  • Windows Flyout RTL isolation: initialize this independent RTL scenario with fresh fixture state, consistent with the existing title isolation. CI8's mismatch contains genuinely mirrored glyphs, not just perimeter drift. The original screenshot and tolerance remain. CI9's first Windows executions pass both this scenario and Issue30536.
  • Windows Map completion and result integrity: port the coordinated Map/runner corrections from [net11] Fix Helix payload and device-test failures #38827. Await native Map scripts before assertions/cleanup, retain the required System import for WinRT await extensions, and reject empty or malformed category results and abnormal process exits. Keep the packaged launcher's process handle until exit-code retrieval. CI9's supposedly successful unpackaged Controls run had a zero-byte Map result, two missing Map tests and an actual native crash dump; its readable aggregate was not proof of completeness.
  • Android SwipeItem drawable ownership: call Mutate() before changing the icon's bounds or tint. Resource drawables can share their constant-state paint: tinting a SwipeItem black contaminated later Images using the same packaged resource. At the reproduced failure the underlying bitmap was red, but the drawable carried a black filter and both the real screen and direct native draws were black. A native regression checks that tinting the real SwipeItem leaves resource drawables loaded before and afterward untinted. The original undefined-size/background Image regression is unchanged. After independently reproducing and fixing this, branch comparison confirmed that [NET 11] SwipeItem: Add explicit IconColor and TextColor #36884 already isolates tint state on net11/RC2 as part of the new IconColor/TextColor API; this minimal main-branch correction does not backport that public API or change main's tint policy.
  • SafeArea Grid orientation observation: observe the requested orientation, current screen axes, insets and all required edge positions together through the existing NUnit-isolated assertion polling helper. An orientation-command acknowledgment does not mean the safe-area layout has settled. Rotate once outside each observation and restore portrait in finally. Every original position/roundtrip assertion and the one-pixel tolerance remain.
  • Mac Issue26598 diagnostics: capture read-only native tab-bar state for the existing HostApp UI failure, preserve the capture before fixture teardown and restore logging configuration. CI10 establishes an actual native visibility reversal on macOS14.8.7: the bar is visible at Navigated, then hidden 500ms later despite ShowTabs=true and the top controller's HidesBottomBarWhenPushed=false. The Mac15/26 native passes do not exercise this pre-18 UIKit branch. The next diagnostic observes changes to the tab layer's hidden state and records native/managed caller stacks plus every navigation controller's flag; it is disposed on Shell unload. Neither diagnostic changes native visibility or test assertions.
  • Legacy UIKit Shell tab bar (Issue26598 product fix): CI11's caller-stack diagnostic shows the hidden Home root setting the root ShellSectionRootRenderer.HidesBottomBarWhenPushed=True; MAUI unhides the tab bar for the pushed InnerTab page, then UIKit re-hides it with no managed MAUI frame on the stack. Before iOS/Mac Catalyst 18, ShellPageRendererTracker hides tabs through HidesBottomBarWhenPushed, and legacy UIKit hides the bottom bar after a push when any controller in the resulting stack requests it. ShellSectionRenderer.PushPage now clears those stale requests from the current stack (or the More navigation stack) before pushing a page that wants the tab bar visible, on pre-18 only. Pops still hide tabs for hidden targets because Shell sends Appearing, which re-applies the target's flag, before native pops. iOS18+/Catalyst18+ are unchanged. A local iOS18.5 run with the legacy branch forced on did not reproduce the Mac failure, so legacy Mac CI is the validation lane.
  • Android emulator ADB identity: CI11's Material3 bootstrap rotated the host ADB identity every 60 boot iterations and attempted a device push before launch, producing unauthorized emulators. Keys are now prepared once on the host without rotating the private identity, adb pubkey restores a missing public key, the private key is used as the vendor key, and setup errors propagate. Eleven isolated Pester cases cover the behavior. This does not claim to fix the separate CoreCLR Entry job that had no attached emulator.
  • Issue32983 sheet fixture lifetime: keep the managed sheet content alive while its native controller is presented. The native controller and legacy CollectionView hold weak managed references; the unretained fixture reproduced an ItemsViewController NullReferenceException and native abort on iOS26.5. The measure-before-mount scenario and screenshot oracle are unchanged.
  • Bugzilla30353: after the final portrait rotation, wait for DetailToggle before its once-only tap. The CoreCLR request log shows rotation returning before both button queries returned empty. All IsPresentedChanged assertions and wait budgets are unchanged.
  • Appium log retention across retries: archive the previous server log before a retry starts another Appium process, so category retries no longer overwrite the first failure's request log.
  • Binlog fixture failure diagnostics: retain captured child-process output when fixture creation returns an unexpected exit code. CI9's Windows timeout previously reported only 1 != -1, discarding that output. The sixty-second timeout and expected exit code remain unchanged. This does not claim to fix the still-unexplained child-process timeout.

No new quarantines, blanket test retries, tolerance increases, or resized replacement baselines are introduced. Existing platform exclusions are not broadened.

Validation

Draft: the CI-to-green loop is still in progress. The latest correction batch still requires complete current-head CI confirmation; this PR is not verified green. Local UI validation uses isolated iOS simulators and explicitly selected Android emulators only. MacCatalyst and Windows runtime validation is CI-only.

CI12 published (f988de7621, merge 2320d99689): adds the legacy UIKit Shell tab-bar correction, Android ADB identity, Issue32983 fixture lifetime, Bugzilla30353 wait and Appium log retention. All three pipelines are triggered; no current-head outcome is claimed yet.

Completed CI11 (900b6f544c, merge 8447dc3ddd): main1611982 and native1611987 succeeded (native: all 46 payloads, 29,394 passes, 1,038 existing skips, no failures; main Build integration passed 93/93 on both macOS and Windows). UI1611986 finished failed. Its two terminal published failures are iOS-latest CV1 Issue32983 (missing sheet items; lifetime corrected in CI12) and API30 CoreCLR Bugzilla30353 (first execution passed, category retry failed its post-rotation tap; wait corrected in CI12). The legacy Mac Shell first attempt completed 336 tests with 322 passes, 13 unrelated skips and one failure, Issue26598; the category retry then hit the job limit. All 67 original-scope pairs in that job are MacCatalyst: 66 passed first execution and Issue26598 failed; its diagnostic identified the root cause fixed in CI12. At least 60 first-attempt failure occurrences are retained from job logs even where retries passed: 48 of 49 Windows occurrences have exact same-method/same-condition failures on main, while Windows CV5 VerifyGroupHeaderAndFooterTemplate_WithObservableCollection, Android Issue11214 (different phase from main), Mac Issue17389 input transparency and Mac Bugzilla59172 remain unmatched and unresolved; no speculative changes are made for them.

Completed CI10 (666d00b8a5, merge a526ee7bee): all three pipelines finished failed. Main1611811 published 18,046 passes and 140 not-executed results. Windows Build integration passed all 93 cases, including the four binlog utility facts; macOS Build lost agent AcesShared61 and its result/log publishers were abandoned, leaving those 93 cases unverified. There is no established source fix or proven cause for the agent loss.

Native1611812 has all 46 readable latest payloads: 29,393 passes, 1,038 existing skips and one failure, MemoryTests.BindableLayoutDoesNotLeak on iOS18.1, retaining one Ellipse. The exact latest-main1608679 iOS18.1 case passes. The unchanged test passes locally on iOS18.5 in the Memory class, the complete 618-case General context, and five fresh Memory processes; this does not reproduce or explain the CI18.1 failure. Both Android Swipe/Image suites, both Essentials payloads and both Windows Map configurations now have actual passing results. The original Mono Controls attempt's DEVICE_NOT_FOUND81 setup failure is retained despite its later pass.

UI1611813's 151 published result sets contain 17,911 passes, 295 unrelated not-executed results and two failures: API30 Issue32223 and Mac Issue26598. Original scope is 124 passing pairs, one failed Mac pair and one mixed Android pair; expanded coverage is 146/148 passing configurations, two failures, none missing. The Android failure reproduces locally and is corrected in CI11. Mac Shell failed both first execution and retry; the complete first result set is 322 passes, 13 unrelated not-executed and one failure, while the retry hit the three-hour job limit. Native logs demonstrate actual tab re-hiding on macOS14.8.7, not merely a stale accessibility locator. The new caller-stack diagnostic is intended to identify that still-unproven cause.

Issue2951 passes all five observed CI10 configurations, including the first Mac execution. The corrected SafeArea roundtrip passes its published iOS-latest and dedicated AndroidAPI36 configurations. Mac Issue17389's functional and visual cases and Issue16918's unchanged golden pass first execution; their historical failures remain recorded and are not claimed causally fixed. The available UI logs retain 59 job-level failure occurrences, including first attempts and category retries, after deduplicating task-log copies; later published passes do not erase these failures.

CI9 (244f001d8a, merge f46610fb39): main1611690 finished failed: Windows BuildWarningsUtilitiesTests.RejectsTruncationAfterBuildFinished(False) timed out creating its tiny fixture binlog; the other 92 cases in that integration leg passed. The timeout's cause is not established. Native1611700 finished failed: all 46 latest aggregates contain 29,389 passes, 1,038 skips and one Android Image failure, but the Windows Map category is incomplete. Its two absent tests, zero-byte XML and native crash remain failures despite the enclosing runner's success exit. All 46 native consoles/artifact inventories were audited, not merely the aggregate counts.

The same Windows binlog-fixture exit-code signature occurred before this correction in unrelated inflight/current PR #38614: build1604992/log1025, September 21, expected exit 1 versus actual -1. This was verified in the raw log, not inferred from another review. The latest completed main build1608677 has actual passing results for all four binlog utility cases (Windows run44477112). The historical occurrence does not establish the child timeout's cause or make the current failure green.

UI1611699 finished failed. Its 151 published result sets contain 17,909 passes, 295 unrelated not-executed results and four terminal failures. Original scope: 125/126 observed passes; expanded configurations: 147/148 observed passes, with no missing scoped pairs/configurations. Mac Issue26598 remains the one failing original-scope pair; both attempts failed the initial visible-tab assertion. The other terminal failures are the iOS SafeArea roundtrip, Mac Issue17389's pre-toggle tap-count timeout and its separate background-color screenshot mismatch. Mac Layout and Shell retries hit the three-hour job limit; their complete first-execution results were recovered, not mistaken for complete retries. All 26 Android and 13 Windows original-scope pairs passed. The Mac Image category recovered on its task retry, but the first Issue16918 snapshot failure remains recorded: the actual capture contains a full-width separator and icon-rendering differences, not merely window-border noise. No replacement golden or speculative correction was made.

Local evidence for the follow-up batch: the new SwipeItem ownership test fails inside the post-tint assertion on both API30 and API36 before the product correction (20 passed/1 failed per focused run), then all 21 cases pass on each with Mutate(). Broader native context passes on both devices: 858 passed, 11 existing skips, zero failures/fixture errors out of 869 cases. API36 executes SwipeView before Image, explicitly covering the order associated with the original failure; all eleven original Image cases pass unchanged.

The unmodified SafeArea roundtrip fails on iOS18.5 while observing transitional landscape geometry. The correction passes twice there, alongside all four assertion-polling guard tests; the target and three neighboring Grid edge cases also pass on iOS26.5 and Android API30. The stable-identity Issue2951 correction passes two Android runs and one iOS run. The Windows runner port passes sixteen actual old/new XML checks; the exact WinRT await expression reproduces CS4036 without System and compiles with it. A full Windows project build is blocked by MakePri.exe on macOS and is not reported as successful validation. The binlog utility's four existing tests pass before and after its diagnostic-only edit using the installed .NET10 SDK.

Completed CI8 (94a72b791a): main1611454 succeeded. UI1611456 finished failed with 149 published result sets: 17,728 passes, 295 unrelated not-executed results and three terminal failures. Original scope: 125/126 pairs passed, one Mac Issue26598 failure, zero missing pairs or scoped skips. Expanded configurations: 146/148 passed, one Mac failure and one unexecuted CoreCLR/API30 Kiwi configuration. The three terminal named failures are Windows Issue30536, Windows Flyout RTL and Mac Issue26598. Mac26598 failed both executions at the initial visible-tab assertion and its Shell job hit the three-hour limit; current screenshot and native hierarchy confirm that the bar is physically absent. Mac Issue2951 additionally failed its first execution before the first click with a stale native reference; the existing task retry passed, which does not erase that failure. Mac Layout's complete first execution passed 189 cases with four unrelated skips, including Issue24996; Search/Shape/Slider passed 172.

Two CI8 UI jobs never executed tests: CoreCLR/API30 CollectionView3 failed package installation with exit78 after a broken pipe/device-offline sequence, and iOS CV1 CollectionView4 failed downloading the SDK releases index with TLS EPROTO/internal-alert80. Build Analysis did not match these failures. They are not counted as passing or asserted to be proven infrastructure problems. The missing expanded-scope case is the CoreCLR/API30 ByIndex/End/horizontal Kiwi case; its Mono counterpart passed.

CI8 native1611455 finished failed. All 46 latest XML payloads were read: 29,391 passes, 1,038 skips and one named CoreCLR ScrollView failure, with no fixture/count-integrity errors. That failure has empty living-reference details and matches unchanged-main1605580. Original Essentials Mono and CoreCLR crashes still count despite passing later attempts. Both Mac full native Issue26598 cases and all four Apple initial-transition cases passed; the native tests replace the root in an existing window and do not establish correctness of HostApp's initial-window UI flow. GitHub Actions run36067589906 passed 2,268/2,268 with no skips.

Local evidence for the five-commit 244f001d8a batch: the GC unit proof changes from 6 passed/2 failed before to 8/8 after, and the adjacent selected suite passes25/25. Recreation fixtures pass 36/36 across API30/API36 on the final APK; baseline also passed, so the CI crash is not claimed reproduced locally. The Windows client compiles with zero warnings/errors. No desktop application was executed locally.

The final GC helper also preserves real native-retention failures: normal Mono/API36 passes33/33, but Mono/API30 and CoreCLR JIT/API36 each fail DoesNotLeak with three genuinely retained objects. An exact old-helper Mono/API30 control produces the same failure types and geometry; only per-process native identity tokens differ. A post-failure CoreCLR heap shows strong native GC handles to MauiScrollView and handler retention through native content/listener objects. This is distinct from the empty-diagnostic verdict bug, remains unresolved, and does not justify the rejected JVM/bridge collector proposal or a speculative product fix. Local JIT/ARM64 is not CI ReadyToRun/x64 equivalence.

The preceding ff9459cbde UI run published 56 complete result sets: 6,969 passes, 107 unrelated not-executed results and ten Windows failures. Within the original scope, 35/126 pairs passed and 91 were unpublished (35/148 expected configurations observed). All 13 Windows pairs and 22 Android pairs passed; Apple and remaining Mono configurations were blocked by a one-hour Mono sample-build timeout. Windows Issue18896 and Material3 Issue35119 each passed their first execution with unchanged screenshots. The Windows multi-window pointer assertion failed on the category retry; its failure skipped secondary-window cleanup. Nine following screenshot mismatches were almost entirely native window edges (seven had zero differing interior pixels beyond a five-pixel perimeter; two had four and nine). This correlation does not by itself prove causality. The earlier Editor autosize comparison failure remains recorded; only its teardown screenshot survived.

That head's native build timed out during Mono BuildTasks. The 15 available payloads contained 10,602 passes, 408 skips and no named, fixture or count-integrity failures; 31 expected payloads, including Apple coverage, were unpublished. All seven CoreCLR gesture variants passed first attempt without any collector/product change. Essentials CoreCLR passed attempt two; its earlier XML was unavailable. Main's Mac pack reported an unmatched MSB4166 child-node failure and Release BuildTasks remained stuck until the next push canceled the build. Same-head native retry1611437 also became superseded before publishing Helix jobs. These are not passing evidence or proven infrastructure attribution. GitHub Actions run36043714050 passed 2,268/2,268 with no skips.

The Windows pointer correction compiles with zero warnings/errors and passes targeted formatting checks. The native driver's documented command and implementation explicitly move the cursor before clicking; the coordinate-space correction is described above. No Windows application was executed locally; actual pointer behavior and cleanup require CI evidence.

The completed 5293fcaf73 iteration observed 125/126 original method/platform pairs passing, one Mac Issue26598 failure, zero scoped skips and zero missing pairs. Across the 148 expected configurations this was 147 passes and one failure. Main succeeded. Device XML contained 29,383 passes, 1,038 skips and seven failures: six CoreCLR gesture-memory cases and the baseline-matched Mono Image failure tracked by #38740. All 46 native payloads were checked, with no fixture/count-integrity errors. The new first-transition case passed in all four Apple lanes; that does not prove the full UI flow.

Mac Issue24996 passed first execution in run44547244/result100063 (10.78 seconds; its Layout run had 189 passes and four unrelated skips). Mac Shell's first execution had 322 passes, the Issue26598 failure and 13 unrelated skips; its retry repeated that failure before the three-hour job limit. A separate Mac SearchBar/Shape/Slider job was canceled after discovering 172 cases and published no TRX; its cause is unproven and that lane is not counted as passing. Retry-hidden non-scoped failures remain recorded, including the Material3 prompt cancellation and Windows Issue18896 comparison addressed in this batch. GitHub Actions run36003200959 passed 2,268/2,268 with no skips.

Local evidence for ff9459cbde:

  • Material3 Issue35119 passes three independent first-attempt full NUnit executions, each retaining all three original styling comparisons. On the exact final APK, the old keyboard command prematurely cancels the prompt under a controlled slow-IME transition, while the single-ESC sequence retains it until explicit Cancel in five corresponding trials. Emulator animation settings were restored. Local Appium/architecture differ from CI; its subsequent first-attempt CI pass is recorded above.
  • The Windows UI test assembly compiles with zero warnings/errors. Its scroll assertions read the native UIA source, fail on missing state and retain the original golden. Its subsequent first-attempt Windows CI pass is recorded above.
  • The final existing Shell suite passes 227/227 on each of iOS18.5 and iOS26.5, with no skips or fixture errors. The full native-root helper was separately exercised through a temporary iOS18.5 entry point and passed; that temporary entry point is not shipped. The new Mac-specific case compiled with zero warnings/errors and subsequently passed both Mac CI8 lanes. A broader iOS26.5 probe still reports a later hidden-page mismatch (TabBarHidden=true while the bar remains hittable); it is not claimed fixed, and no existing iOS test is excluded.
  • Four bounded, unprofiled memory experiments rejected the proposed JVM/CLR GC-oracle correction: it did not produce clean CoreCLR positives. A strong-held/released control passed on both runtimes, but ordinary realized-label cases still failed CoreCLR. All experimental collector/product edits were restored; no speculative collector or product fix is included. The later final-liveness verdict correction is separately described and verified above. Local CoreCLR JIT/ARM64 is not the CI ReadyToRun/x64 configuration.

Earlier local evidence includes 226/226 native Shell tests on iOS26.5, 15/15 AbsoluteLayout tests on iOS18.5, 15 AndroidAPI30 Shell UI cases, AndroidAPI36 CollectionView/navigation scenarios, and focused iOS18.5 Issue20855/26598/29588/7045/30771 cases. Original applicable goldens remain unchanged. Runs accidentally selecting a stale host on another Android emulator were discarded and rerun with explicit device logs verified; missing baselines were not fabricated.

For the preceding correction batches:

  • Issue35736's four original UI cases pass first attempt on each of iOS18.5 and iOS26.5: 8/8.
  • After Google Search isolation, the five affected Android Shell cases pass first attempt on the verified owned API30 emulator with freshly rebuilt/deployed APK hashes. Search stays disabled after reboot and cannot autostart; Play Services/Store/Maps remain enabled. Ten script safety/error cases and Cake compilation pass. The exact CI version-triggered redirect was not reproduced locally; the identified producer was demonstrably blocked. Both Android Shell CI lanes subsequently passed on 8e026640a8.
  • Headless assertion checks exercise the actual compiled Windows RTL helper against real CI XML and negative geometry variants (7 checks), the actual compiled Mac selection/color helper (8 checks), and source-linked scroll boundary/mobile-extent cases (7 checks). These do not substitute for desktop CI execution.
  • Issue9088 passes in five fresh iOS26.5 sessions and one iOS18.5 session with no skips.
  • Eight headless checks exercise the compiled Shell transition helpers: reproduce the old stale-reference failure, require a fresh visible element, reject false absence and persistent wrong state, preserve unexpected errors, and match the actual CI back arrow rather than the same-name tab.
  • The new Shell helper batch passes 7/7 focused iOS26.5 cases on the explicitly selected simulator, with no failures, skips or test retries: Issue26598, all four Issue32994 visibility cases, and both animated/nonanimated presentation cases.
  • All 112/112 tests in the three affected script-gate fixtures pass locally after reproducing the three CI failures. Reversing and reapplying the trusted screenshot patch reproduces the current harness byte-for-byte. The full GitHub Actions run35975315118 on 51df615d9e passed 2,268/2,268, with no skips.
  • The lifecycle correction passes all six ActivityStateManager/recreation cases in four fresh AndroidAPI30 launches: 24/24, no failures/skips/fixture errors. The first API36 invocation failed before tests because XHarness's default instrumentation user was rejected; it is not counted as a pass.
  • Issue6994 and Issue2951 pass in three fresh CoreCLR JIT sessions and one normal Mono session on AndroidAPI30: 8/8, with actual runtime libraries verified. Local CoreCLR ReadyToRun failed before tests because the installed crossgen tool rejects Android; only the local validation process disabled ReadyToRun. CI's normal configuration is unchanged and still required.
  • Eleven source-linked scroll checks reproduce the old premature deadline failure and exercise slow progressing traversal, exact final-gesture success, over-limit/missing/clipped failures, and unchanged mobile/desktop extents. The actual Issue29588 AndroidAPI30 UI test passes; Mac/Windows test assemblies compile. These headless checks are not desktop runtime validation.
  • The corrected ScrollViewNoContent completion assertion passes on the owned AndroidAPI30 emulator and iOS26.5 simulator, with no skips or retries. Desktop runtime confirmation remains CI-only.
  • Issue24996 passes all four original translation positions on iOS26.5 and AndroidAPI30. A temporary forced ancestor width change fails the iOS assertion with Step 1: Lvl1[1/1]; that injected change is not included in the patch.
  • The focused native Issue26598 case passes within 227/227 Shell tests on each of iOS26.5 and iOS18.5, with zero skips or fixture errors. It targets the currently failing first transition on Mac; the existing full UI scenario remains enabled. An earlier broader native probe found a separate later iOS hidden-page mismatch (TabBarHidden=true while the native bar remained hittable); that is not claimed fixed by this first-transition case.

CI progression:

Head Main Device UI
fbbdd4449b 1609439: passed 1609440: failed 1609438: failed
ec017da74d 1609951: passed 1609953: passed 1609954: five failed jobs; remaining jobs superseded by push
8e026640a8 1610175: passed 1610176: two baseline-matched Android Image failures 1610177: failed; later same-head restart superseded by push
d4815b6101 1610304: superseded 1610305: superseded 1610306: superseded by script-gate correction
51df615d9e 1610340: passed 1610341: lifecycle fixture failed 1610342: Android Button job failed; Mac Shell/Layout retries hit the three-hour job limit
5293fcaf73 1610649: passed 1610656: six CoreCLR gesture failures and one baseline-matched Image failure 1610657: canceled; Mac Issue26598 failed
ff9459cbde 1611219: pack child-node failure; later superseded 1611221: Mono build timeout; 15/46 payloads published 1611222: Mono build timeout and Windows failures
94a72b791a 1611454: passed 1611455: failed; ScrollView and original Essentials crashes 1611456: failed; Windows pointer/RTL, Mac26598 and two unexecuted jobs
244f001d8a 1611690: failed 1611700: failed/incomplete 1611699: failed
666d00b8a5 1611811: failed; macOS integration results missing 1611812: failed; iOS18.1 BindableLayout 1611813: failed; Android32223 and Mac26598
900b6f544c 1611982: queued/running 1611987: queued/running 1611986: queued/running

On 51df615d9e, Windows Issue32223, Issue29588 and FlyoutPage title have actual passing results. iOS9088 executed and passed on iOS26.5 rather than being skipped. The complete original-scope inventory is 125/126 pass-observed, one Mac Issue26598 failure, no scoped skips and no unpublished pairs; it is not green. Mac Shell's complete first execution contains 322 passes, one failure and 13 unrelated skips. Both presentation-mode cases passed. Its partial retry repeats the Issue26598 failure and leaves 142 cases unobserved before the job timeout; the first execution's results were recovered rather than treating that incomplete retry as coverage. All 46 latest native XML payloads were read: 29,385 passes, 1,038 skips, one lifecycle-fixture failure, with no fixture errors/count mismatches and 32/32 new Apple Shell cases passing. CoreCLR's earlier attempt failed before testing with DEVICE_NOT_FOUND; Windows packaged attempt1 remains unavailable while attempt2 passed.

Additional failures on that head were the exactly-one-event native assertion, Issue6994's ListProxy/ListViewAdapter index crash during layout, Issue2951's expected-two/actual-three button observation, and Mac Issue29588's traversal timeout recovered by an existing task retry. The corrections above address those observed fixture/synchronization defects without changing product code. Recent exact baseline samples passed these native/Button cases; #36979 documents the historical Issue2951 assertion, but that is not proof of the present cause. No baseline attribution or retry recovery is presented as a clean first execution.

Later Mac results also exposed ScrollViewNoContent's inverted completion assertion; it failed first execution and passed retry. Five recent main executions passed, but absence-based passes do not prove the scenario completed. Two unchanged Mac CollectionView spacing snapshots recovered on retry; both first-failure screenshots contain an external Finder alert saying WindowManagerShowDesktop-Education is not responding. No spacing, screenshot algorithm, golden or tolerance changes are made for that overlay. No exact known issue was found for it.

Mac Layout's complete first execution on that head contained 188 passes, one Issue24996 failure and four skips. Its partial retry timed out before reaching that case. The failure hierarchy showed the first translation at X40/Y80 with cumulative Lvl1[2/5]; three of the five sampled main runs also failed the same assertion. The fixture correction separates translation measurements from initial/diagnostic layout and requires a fresh result. Local iOS validation deliberately forced an ancestor width change and failed with Step 1: Lvl1[1/1]; removing that injected change passed all four original translation positions. The subsequent 5293fcaf73 Mac first-execution pass is recorded above.

The second iteration's native evidence includes all 32/32 new Apple Shell cases across four configurations, including 16/16 Issue35736 image/reset cases. All seven Helix jobs finished. Reading 46 result XML payloads found no named failures or fixture errors; Windows Graphics attempt1 XML was unavailable, while attempt2 passed. A green XHarness exit code was not used as the sole evidence.

For 8e026640a8, every original scoped pair has actual results: 122/126 pass-observed, three Mac Shell failures, one iOS9088 runtime skip. The three failures are Issue26598's stale tab reference and the two presentation-mode back locators corrected above. This is not a claim of 126 first-attempt passes. Additional failures outside those pairs include Windows Issue32223 and FlyoutPage title, also corrected above; Mac SafeAreaEdges checkout failed and Mac CollectionView3 lost its agent, so those incomplete categories are not counted as passing.

The third iteration's 46 native result payloads contain 29,382 passes, 1,038 skips and two Android Image failures; all 32/32 new Apple Shell executions passed again. The CoreCLR Image failure matches main build1605027/run44338780/result100881. The Mono failure matches main build1600801/run44201018/result100693, including the identical black PNG. Both are tracked by #38740. This was baseline attribution, not a green device build; no speculative Image change was included in that iteration. The later reproduced shared-tint root fix is described above. A fresh owned CoreCLR emulator run passed all 11 Image tests.

Both original Android Label goldens passed in API30 run44515496 (log3511). A separate formatted-to-plain styling issue reproduced in a local source-built framework did not reproduce in this CI product build; its candidate framework fix stays outside this PR. Issue32941 also has a named second-iteration pass. Previous-head passes are not substituted for new-head coverage.

Retry-hidden failures remain recorded separately. The third iteration's 40 Windows gallery setup timeouts match the exact base; Bugzilla51825's separate test-body timeout also matches 1608678/log6414. Two unchanged iOS Issue22306_1 RotateButton timeouts and one unchanged Mac Issue28416 DropResult timeout recovered on retry, but exact current-stack baseline attribution remains incomplete. Historical #33220/#33245 mention the Mac test, without accessible exact-stack confirmation. Earlier Mac Carousel Issue16020 evidence is retained. None of these retry passes erases its initial failure, and no unfinished or canceled category is counted as passing.

Issues Fixed

Addresses flaky test coverage identified in #38777 without adopting its exclusions. Most changes affect test fixtures and infrastructure rather than the framework regressions named by those tests. The focused Android SwipeItem change additionally fixes the empirically reproduced shared-resource tint contamination described above; unrelated unresolved product behavior is not claimed fixed.

Approaches deliberately avoided

  • Do not resize fullscreen captures to make old goldens pass; restore leaked native window state.
  • Do not catch NUnit assertion exceptions without isolating their recorded result.
  • Do not replace native rendering assertions with labels echoing MAUI properties.
  • Do not disable Google Play Services or remove Google APIs to prevent an unrelated Search popup.
  • Do not call all platforms verified based on a passing simulator run or pipeline check color alone.

@github-actions

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 38822

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 38822"

@azure-pipelines

This comment has been minimized.

@kubaflo

This comment has been minimized.

@kubaflo

This comment has been minimized.

@kubaflo

This comment has been minimized.

@azure-pipelines

This comment has been minimized.

2 similar comments
@azure-pipelines

This comment has been minimized.

@azure-pipelines

This comment has been minimized.

@kubaflo

This comment has been minimized.

@kubaflo

This comment has been minimized.

@azure-pipelines

This comment has been minimized.

1 similar comment
@azure-pipelines

This comment has been minimized.

@kubaflo

kubaflo commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

/azp run maui-pr
/azp run maui-pr-devicetests
/azp run maui-pr-uitests

@azure-pipelines

Copy link
Copy Markdown
No pipelines are associated with this pull request.

@kubaflo

This comment has been minimized.

@kubaflo

This comment has been minimized.

@azure-pipelines

This comment has been minimized.

kubaflo and others added 19 commits October 26, 2026 12:00
Extract the existing GC assertion methods without changing their behavior, and source-link them into Core.UnitTests so the shared verdict logic can be tested deterministically without a device.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Do not retain a failed earlier reference observation after subsequent collections have released that target. Preserve the existing collection budget and reject every still-live target. Add deterministic late-collection tests and real retained-object controls in Core.UnitTests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep native listener teardown and fixture reset in the same main-thread turn as activity cleanup. This prevents off-thread disposal from racing the native lifecycle callback snapshot seen in the original Mono CI crash. Preserve all recreation, cancellation, and security-exception assertions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Include the live main-window origin in native mouse coordinates and require observed secondary-window creation and minimization before asserting pointer events. Preserve exact event counts and unconditional cleanup.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Start the existing dynamic RTL scenario with a fresh fixture. Preserve all actions and its original screenshot baseline, tolerance, and timeout.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Look up the intended B/B/C buttons immediately before each input instead of retaining duplicate-ID indexed elements across ListView updates. Keep the count, text-transition, removal and original background-color screenshot assertions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Record existing tab-controller and ancestor visibility without forcing layout or changing the original UI assertions. Scope the native log to this fixture, preserve it before teardown can restart the app, and restore any previous shared logging configuration.

This is diagnostic evidence collection, not a product fix for Issue26598. Local validation is limited to headless compilation and logging-lifecycle checks; native desktop execution remains CI-only.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Port the coordinated Map fixture and Windows runner corrections from #38827 (20632bd, c2f5a5c, 8768da7). Preserve the native-layer assertions, recognize normal WinUI -1 exits, retain packaged process handles, and fail invalid category XML without discarding valid sibling results.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The exact WinUI ExecuteScriptAsync await fails CS4036 without System; the device-test project does not enable implicit imports. Restore the source port import before publication. Headless expression compilation verifies the failure before and success after, without executing a Windows application.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Mutate the drawable before setting bounds and color filters so a SwipeItem cannot tint other Image instances using the same resource. Add a native regression covering drawables loaded both before and after the SwipeItem icon.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Read orientation, insets and all required positions together in the existing isolated assertion polling helper. Preserve the original position and roundtrip assertions and pixel tolerance, rotate only once per transition, and restore portrait in finally.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the expected failure exit code and sixty-second timeout, but include captured output when setup returns an unexpected code. The prior assertion discarded the only child-process diagnostics from the observed Windows timeout.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Drag David beyond Charlie's center instead of stopping exactly at ItemTouchHelper's strict drop threshold. Reuse the existing Windows touch sequence; preserve one real input, the ReorderCompleted event assertion, and geometric item-order checks. Original API30 NUnit reproduced the CI failure; the fixed test passed twice on API30 and once on API36.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Capture native and managed caller stacks when the tab layer changes hidden state, together with all native navigation-controller flags. Dispose the opt-in diagnostic observer when the Shell unloads; leave visibility and test assertions unchanged.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Prepare valid host keys before launch without rotating the private identity, contacting an unbooted device, or generating invalid fallback keys. Restore missing public keys with adb pubkey and surface setup errors. Add eleven isolated regression cases and include the Cake source in the script-test trigger.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the sheet content alive while its native controller is presented. The controller and legacy CollectionView hold weak managed references; the unretained fixture reproduced an ItemsViewController NullReferenceException and native abort. Preserve the measure-before-mount scenario and unchanged screenshot oracle.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The CoreCLR CI request log shows rotation returning successfully before both button queries return empty. Observe the button before the once-only tap, preserving all IsPresentedChanged assertions and existing wait budgets.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Archive the previous server log before starting another Appium process so category retries retain the original failure's input requests. Keep the configured path for the latest attempt and retain archives beside it for existing artifact collection.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Before iOS/Mac Catalyst 18, Shell hides the tab bar through
HidesBottomBarWhenPushed and UIKit hides the bottom bar after a push when
any view controller in the resulting stack requests it. A visible page
pushed above a hidden Shell root was therefore hidden again after MAUI
made the tab bar visible, which failed Issue26598 on the legacy Mac
Catalyst lane.

Clear stale hidden-bottom-bar requests from the current stack before
pushing a page that wants the tab bar visible. Pops still restore the
target page request because Shell sends Appearing before native pops.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@kubaflo
kubaflo force-pushed the kubaflo-reliable-quarantined-tests branch from 1cc3ca3 to bc772c6 Compare October 26, 2026 11:00
@kubaflo

This comment has been minimized.

@azure-pipelines

This comment has been minimized.

kubaflo added a commit that referenced this pull request Oct 26, 2026
Reuse the existing focus-aware keyboard helper after explicit Entry focus so a false early IME query cannot skip dismissal. Preserve the exact event and cursor assertions. Remove the Android Udid addition already owned by older PR #38822.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 35e8b0da-aca3-469d-8dca-b808947cbe81
kubaflo added a commit that referenced this pull request Oct 31, 2026
…39742)

<!-- Please let the below note in for people that find this PR -->
> [!NOTE]
> Are you waiting for the changes in this PR to be merged?
> It would be very helpful if you could [test the resulting
artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from
this PR and let us know in a comment if this change resolves your issue.
Thank you!

<!--
!!!!!!! MAIN IS THE ONLY ACTIVE BRANCH. MAKE SURE THIS PR IS TARGETING
MAIN. !!!!!!!
-->

### Description of Change

<!-- Enter description of the fix in this section -->

Intentionally targets `release/11.0.1xx`. Align SDK dependencies with
their coherent parents and supported Xcode, restore blocked execution,
and fix fixture/readiness defects **without adding exclusions or
weakening assertions**. Published head:
**`a7a065d0597357accfc99af3c75c0fbe7c630287`**. Its new full native CI
is running. **This PR is not wholly green:** Android Debug resources,
Windows/Catalyst UI failures, packaged deployment, Pester policy and
Maestro acceptance remain unresolved.

[marker]: <> (Begin:dd7cfc83-01ef-4e81-9c5f-0275227f2fa5)
## From https://github.com/dotnet/android
- **Subscription**:
[dd7cfc83-01ef-4e81-9c5f-0275227f2fa5](https://maestro.dot.net/subscriptions?search=dd7cfc83-01ef-4e81-9c5f-0275227f2fa5)
- **Build**:
[11.0.1xx-6d4cadfefbeab3fb1bab7a13b968833799fc7c48-1](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15859290)
([338938](https://maestro.dot.net/channel/8298/github:dotnet:android/build/338938))
- **Date Produced**: October 29, 2026 6:00:40 PM UTC
- **Commit**:
[6d4cadfefbeab3fb1bab7a13b968833799fc7c48](dotnet/android@6d4cadf)
- **Branch**:
[release/11.0.1xx](https://github.com/dotnet/android/tree/release/11.0.1xx)

[DependencyUpdate]: <> (Begin)

- **Dependency Updates**:
  - From [37.2.24 to 37.2.29][2]
     - Microsoft.Android.Sdk.Windows

[2]: dotnet/android@446d10b...6d4cadf

[DependencyUpdate]: <> (End)

[marker]: <> (End:dd7cfc83-01ef-4e81-9c5f-0275227f2fa5)

[marker]: <> (Begin:9c8396a8-3960-4c04-a42a-e5d8fd60b14b)
## From https://github.com/dotnet/macios — proposed update deferred
- **Subscription**:
[9c8396a8-3960-4c04-a42a-e5d8fd60b14b](https://maestro.dot.net/subscriptions?search=9c8396a8-3960-4c04-a42a-e5d8fd60b14b)
- **Proposed build**:
[20261029.2](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15860281)
([338937](https://maestro.dot.net/channel/8298/github:dotnet:macios/build/338937))
- **Date Produced**: October 29, 2026 5:59:11 PM UTC
- **Proposed commit**:
[01cf76bb05fb2c0f857102f33e29a55bc6b148c1](dotnet/macios@01cf76b)
- **Branch**:
[release/11.0.1xx](https://github.com/dotnet/macios/tree/release/11.0.1xx)

[DependencyUpdate]: <> (Begin)

- **Deferred updates**: proposed [27.0.12230 → 27.0.12269][3] for
Microsoft.MacCatalyst.Sdk.net11.0_27.0, Microsoft.macOS.Sdk.net11.0_27.0
and Microsoft.tvOS.Sdk.net11.0_27.0 requires Xcode27.1. Retain
**27.0.12230**, matching iOS and SHA
`e4732a6dcad66fe70491ad3f808db1e3f2a04541`.

[3]: dotnet/macios@e4732a6...01cf76b

[DependencyUpdate]: <> (End)

[marker]: <> (End:9c8396a8-3960-4c04-a42a-e5d8fd60b14b)

#### Latest implementation

**a7a065d0597 — bound Android failure diagnostics, not test execution.**
Earlier `4338` UI1656915/raw5844 completes its final failed test run
at01:21, then blocks in `adb logcat -d` until02:42 cancellation. Use
Cake's owned `IProcess` and the existing **30-second ADB command
deadline**; its bounded wait terminates the process. Log a typed
diagnostic timeout and rethrow the original UI-test failure. The
standalone logcat target still fails on timeout. No added retries, test
skips, larger test budgets or claimed emulator-connectivity repair.
Update the canonical CI documentation.

**ae09556e2a7 — wait for native search restoration before one focus
action.** Latest `8324` Windows unpackaged execution passes replacement
and disconnection, but removal fails at the immediate native `Focus()`
call after `PopAsync` (line98), before the managed-focus/event
assertions. Wait for the existing box to be loaded, visible and laid
out, then assert the same box and perform the original single focus
action. Preserve all three cases, native-focus checks and the second
focus-event assertion. **New native execution is required; formatting is
not runtime proof.**

**8324f1db042 — remove the delayed strong CarouselView capture.**
`ItemsViewController2` stores its view weakly, but
`UpdateInitialPosition` previously captured a strong `carousel` local
across its existing100ms delay and queued UI callback. Reacquire the
weak view inside that callback and verify its connected handler still
owns this controller. Preserve100ms, all current-item/position
restoration from #36287, every original memory assertion and all40 GC
cycles.

Before: Device1656984/BOT-0306 and Device1657049/BOT-0324 both fail the
generic CarouselView weak-view test on macOS15.7.1; each Controls
run1467/1428 Passed/1 Failed/38 recorded exclusions. After: exact `8324`
Device1657200 passes **all10 Catalyst XML work items on OS15/26**; the
generic CarouselView case passes22.5092ms/14.080ms, and Flyout/Tabbed
variants pass. Actual Apple compile/raw63 succeeds with zero errors.
These are source-labeled native before/after cohorts, **not a
fix-reverted-red/restored-green experiment or full latest UI-position
certification**. Local Xcode26.6/missing maccatalyst workload was not
bypassed.

**cc624e35a36 + ff8f346 — Windows search-focus ownership and real
navigation tests.** Clear detached SearchHandler focus on
removal/replacement/disconnection, null disconnected ownership and
synchronize an attached handler from native AutoSuggestBox/TextBox
focus. Old `4338` TextColor actual46123750/100196 has pixel-identical
red search glyphs but a blank focus marker;195 marker-region pixels
and479 other pixels differ, so the marker is not claimed as the only
difference. No golden, mask or tolerance change. #35079's existing
Show/HideSoftInput wiring is already present, not a duplicate fix.

The first two new tests incorrectly invoked an unchanged CurrentItem
mapper, which returns without retargeting. Replace that setup with real
page PushAsync/PopAsync, retaining all oracles. Both native Windows
builds/raw61,100 pass at `8324`; current unpackaged
replacement/disconnection pass, removal still requires the new readiness
correction. The latest Windows UI TextColor method no longer appears
among failures, but three other Shell screenshots fail; **the complete
visual fix is not certified**.

**bbb5040f6b6 — remove a disproven Catalyst workaround, not coverage.**
Both original relative-coordinate/native event traces and screenshots
put the click inside the visible foreground RedGrid, yet the first
expected tap count1 remains0. Remove optional-target dispatch/private
helper and retain shared physical TapElement, all eight actions,
strict1/2/3/4 then4/0, original readiness and deadlines. Managed
transport/discovery checks are not native success. **The original
gesture/#39775 remains unresolved.**

**4338ff9e1cf — offline picker-scroll fixture.** Use bundled
dotnet_bot.png and remove the separate internet gate that replaces the
entire page with NoInternetAccessLabel. Preserve all17 items, varying
heights40–110, width60, scroll/picker/layout actions and both5px
assertions. Task-owned API30 native scroll→picker Cancel→layout twice
has physicalY1532 unchanged; gate-only reintroduction loses
ScrollButton, restoring the fixed APK restores success. These are manual
native counterfactual measurements, not NUnit certification. Earlier
iOS18.5 run46122258 is59/58 Passed/1 existing exclusion; exact
picker/layout100026 passes4.194s.

**ed65edb9fe0 — previous-.NET coherent children.** Retained macios
parent e4732a6d declares all four **net10.0_27.0 /27.0.10725
/735cfa26761f4478667a0a3d14e32384a1e73fd2** children. Update package
IDs/versions/SHAs, property names and feed together; keep every
coherent-parent relationship. Child Make.config requires Xcode27.0,
packages exist, secure XML comparison matches. `darc verify --ci` still
reports the same four unrelated baseline metadata errors
(TizenUIExtensions/XUnitAnalyzers duplicates; XHarness case/version),
not a pass.

**8786c793965 — normal target merge without new quarantines.** Reject
#39771's added Catalyst gesture and Windows Shell
TextColor/CharacterSpacing exclusions, preserve existing platform
guards. No rebase, squash, force-push or weakened assertions.

#### Latest validation and source identity

| Pipeline | Actual source | State |
| --- | --- | --- |
|
[Main1657387](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1657387)
| Latest `a7a065d`; real merge8053794f, exact pr.sourceSha verified |
Full validation running |
|
[Device1657386](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1657386)
| Exact latest `a7a065d` | Full native validation running |
|
[UI1657296](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1657296)
| Prior `8324` | Still running; classified failures below |
| Latest `a7a065d` UI batch | Automatic batching after prior cohort | No
duplicate manual dispatch |
| [Prior
main1657201](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1657201)
| `8324`; real merge21ee38dc, source verified | **Completed Failed**,
Android Debug APT2260 |
| [Prior
Device1657200](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1657200)
| `8324` | **Completed Failed**, complete9-queue inventory |
| [Earlier
main1656741](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1656741)
| `4338`; real mergee3821933 | **Succeeded**, including Windows
integration |
| [Earlier
Device1656740](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1656740)
| `4338` | **Succeeded;9 queues/62 real XMLs audited** |

**Complete prior `8324` inventory:**9 terminal queues/62 real items;
**61 nonempty result sets/1 missing**. Measured subtotal **42,954
total/41,768 Passed/1 Failed/1,185 recorded exclusions/zero XML error
nodes**. Missing packaged Controls results are null measurements, not
executed tests or zero failures. The strict non-Windows subset is8
queues/52 XMLs: **36,992 total/36,096 Passed/896 recorded
exclusions/zero failures/errors**. These are not relabeled as
newest-head results.

Historical complete inventories remain retained: `cc624e`43,532/42,325
Passed/5 Failed/1,202 exclusions with1 missing;
`bbb`43,801/42,585/4/1,212 with1 missing. Earlier green `4338`
is43,844/42,628 Passed/1,216 exclusions/zero failures/errors, all62 XMLs
after one unchanged failed-Windows-job replacement. That verifies
replacement-worker execution, not repair of corrupt workers. No
all-users runtime/registry cleanup was authorized.

Every initial cohort retains default categories/stages/queues. Queue
controllers and existing exclusions are not counted as executed tests.
Preserve initial failures and missing measurements; no retry-until-green
loop. Certification requires actual terminal identities, nonempty
parseable XML, consistent counters and error-node checks—not XHarness
exit zero or a green badge.

#### Current and historical failures remain explicit

- **Android Debug resources, prior main1657201/raw201:**20 APT2260
occurrences in Controls.Core Android styles. Original failing/passing
binlogs have identical163-item full Controls restore inputs, including
Android37.0/dependencies/asset flags, and both generate
g.props/g.targets and write assets. The failing later build imports zero
package children versus98 in the passing evaluation. Embedded generated
NuGet files capture the same initial portable-only version in both; they
do not prove the later physical file's contents. **No safe root
cause/source patch established; no guessed package reference,
restore-force, SDK downgrade or resource bypass.**
- **Windows native, prior Device1657200/raw165:**unpackaged
Controls896/864 Passed/1 Failed/31 exclusions; removal fails native
Focus() after PopAsync, replacement/disconnection pass. Packaged
Controls on a000ONX fails Appx80073CF0/80070003 before tests; no XML.
Current listed runtime manifests print true, unlike earlier
missing-folder workers, so do not claim an orphaned runtime without
exact deployment-path proof. Eight other Windows apps have nonempty XML
with zero failures.
- **Windows UI Shell2, `8324` UI1657296/raw5671:**both attempts197/181
Passed/3 Failed/13 existing exclusions: CancelButtonColor0.76%,
CharacterSpacing0.76%, TextTransformUppercase0.70%. TextColor is not
among these failures. Native state/actual images must be examined before
any visual repair; no quarantine or golden refresh.
- **Catalyst WebView30381, `8324` UI1657296/raw5734:**both attempts47/46
Passed/1 Failed/zero exclusions;45s Appium timeout tapping
UpdateStatusButton at line28. A separately owned isolated bundled-HTML
fixture/path fix now has retained iPhone/Catalyst evidence; evaluate
only those commits, not promotion ancestors, and do not duplicate
implementation or claim this PR includes it yet.
- **Android CV2, `8324` UI1657296/raw5025:**initial77/76 Passed/1
Failed/zero exclusions, Issue32223 reorder/DataTemplateSelector expects
Success but is empty. Stock full retry77/69 Passed/8 Failed/zero
exclusions; the eight failures are EmptyView fixture setup waiting
EmptyViewButton at GalleryUITest:61, not eight demonstrated EmptyView
product defects. Published run46132128; native root-page state remains
unproven.
- **Older `4338` UI1656915:**Shell2 TextColor0.70%; Catalyst Layout
first RedGrid tap remains red; Shadow fails DNS downloading the MSBuild
logger before tests/no TRX. Android CV3 first117/56 Passed/61 Failed,
stock retry117 Failed after device loss; logcat then blocks82min.
Original results and lack of execution are retained, never turned into
successes.
- **Pester policy:**prior `8324` Actions41157612663/job124407768015
raw2663/2662 Passed/1 Failed/zero Skipped/Inconclusive/NotRun.
Review-Tests.Tests.ps1:85 expects inline GPT5.6 while the workflow
intentionally imports GPT6.1 (#39036). No model downgrade, fake literal,
deleted/skipped assertion or pipeline-test edit. The exception to the
user's pipeline-test-edit prohibition was not approved; unavailable
input is not permission. Newest-head outcome still needs inspection.
- **Maestro:**prior `8324` coherency check124408762176 quotes obsolete
_26.5 children/proposed parent01cf76bb absent from current manifests.
The local graph matches retained e4732a6d; that is not service
acceptance. No parent attributes, pins, subscriptions, channels or check
policies are removed to clear a badge.

#### Earlier implemented fixes and measured evidence

**Historical, source-labeled receipts—not newest-head certification:**

- Retain Android37.2.29/Apple27.0.12230 and supported Xcode; defer
proposed Apple27.1 requirements, never bypass native validation.
- Material3 upstream38c7492 (#38822), three obsolete Switch Ignores
removed (#19883), upstream image32cc36e (#39736): Android29 run46074134
**411/411 Passed, zero exclusions**. No tolerance/mask/crop change.
- WeakList non-inlined factory retains liveness/count/content oracles:
Core Release/Debug1074 Passed/3 existing exclusions each;10 fixture
runs30 Passed/zero excluded, strong-root negative control red; actual
Windows XML passes all three methods.
- Navigation stress preserves10 push/pop cycles/final return, clamps
float progress/ends timers: Catalyst46061170 **72/72 Passed**,
exact347.166s.
- Rotation/keyboard predicate readiness
retains15s/500ms,1px/portrait/keyboard checks: SafeAreaEdges46061290
**164/164 Passed**. Isolated Grid6e47ece: iOS18.5 run46082510 **173/173
Passed**, orientation2.153s/keyboard5.643s; bad-geometry controls remain
red. Not a native revert experiment.
- Checkout-local Appium npm cache from #39600, no global
deletion/chown/elevation/version/retry change; actual CV1 run46055548
**76/76 Passed**.
- Query cleanup #39359/#39725 removes leaked A;three original789×563 Mac
baselines aligned, all four native-icon methods retained:46074330
**149/149 Passed**.
- Depth-one/no-tags UI/integration checkout leaves build/tag versioning
unchanged; job-local CLI state preserves NuGet/Arcade imports.
Missing-TRX publication fails explicitly. No pipeline/configuration
tests written or modified.
- Older bounded recovery receipts remain on their original source:
Android integration46083082 **11/11 Passed**, CV6 **90/90 Passed**, full
main1655682 succeeds. Failed Blazor launch/deployment/tooltip/iframe
cases remain source-labeled, not claimed as causal fixes or latest
coverage.

**Validation commands/results:** `.dotnet/dotnet cake
eng/devices/android.cake --dryrun --target=logcat --boot=false
--create=false` **passes compilation/dry-run with JDK17; no
tasks/devices execute**, existing addon/unused-variable warnings remain.
`.dotnet/dotnet format whitespace --folder --include
src/Controls/tests/DeviceTests/Elements/Shell/ShellTests.Windows.cs
eng/devices/android.cake --verbosity quiet` and `git diff --check`
**pass**. Actual prior `8324` Apple/raw63 and Windows/raw61,100 native
builds pass. Local maccatalyst library build remains NETSDK1147/missing
workload and Xcode26.6 below SDK27; local Windows MakePri.exe cannot run
on Mac. No bypass, local host-macOS XCTest or physical Apple device.
**Newest native runtime validation is pending in the full CI runs
above.**

### Issues Fixed

<!-- Please make sure that there is a bug logged for the issue being
fixed. The bug should describe the problem and how to reproduce it. -->

Addresses SDK/child-coherency blockers, offline picker gating,
bootstrap/readiness/GC/timer defects, query-cleanup images, silent
missing-result publication and unbounded Android failure diagnostics.
Preserves Switch/search-icon/Catalyst-gesture/Windows-Shell coverage.
Native CarouselView retention now has before/after receipts, but the new
Windows readiness/diagnostic changes and complete newest matrix remain
unverified. **Do not claim #39775 fixed. No separately filed issue.
Remaining functional/build/deployment failures, Pester permission and
Maestro verdict keep this PR incomplete.**

<!--
Are you targeting main? All PRs should target the main branch unless
otherwise noted.
-->

---------

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: Jakub Florkowski <kubaflo123@gmail.com>
kubaflo added a commit that referenced this pull request Nov 3, 2026
…39787)

<!-- Please let the below note in for people that find this PR -->
> [!NOTE]
> Are you waiting for the changes in this PR to be merged?
> It would be very helpful if you could [test the resulting
artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from
this PR and let us know in a comment if this change resolves your issue.
Thank you!

<!--
!!!!!!! MAIN IS THE ONLY ACTIVE BRANCH. MAKE SURE THIS PR IS TARGETING
MAIN. !!!!!!!
-->

### Description of Change

<!-- Enter description of the fix in this section -->

A narrow servicing **test-fixture fix** targeting `release/11.0.1xx`,
following #39742. Source `2c049b1fcd078270014484aadef62da0d1110fd4` has
the exact release commit `df9c5fd8109abf9411146d2fe041a2d8967e64ed` as
its direct parent; there are no promotion imports or dependency changes.

`Flyout_BackButtonPressed_Handled_True_Should_Stop_Navigation`
previously performed back immediately after an asynchronous Detail2
push, then started asynchronous cleanup before reading the root
handled-status label. The observed failure was a null reference at that
immediate root-label read, not an established product back-dispatch
failure.

Move the existing Detail2 readiness wait before the back action. Expose
the **same view model's actual event-derived `BackButtonHandled`** on
the current Detail2 page, only when `ShouldHandleBackButton` is true.
Assert the initial `False` value, then assert Detail2 is still displayed
and the actual status became `True`, before programmatic cleanup.
Missing status explicitly fails equality.

Only `FlyoutPageFeatureTests.cs` and `FlyoutControlPage.xaml.cs` change.
The callback, default False value, event/handled-false tests, existing
root labels, platform guards, case/category counts, waits, deadlines,
retries, and all PNG bytes remain unchanged. Ordinary Detail2 snapshot
scenarios do not receive the extra label.

**Native qualification — exact source, unchanged full/default
`maui-pr-uitests` definition 313:**

```sh
az pipelines run --id 313 --branch refs/heads/kubaflo-flyout-back-fixture --commit-id 2c049b1 --org https://dev.azure.com/dnceng-public --project public
```

No parameter or variable overrides were supplied. In [build
1658186](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1658186),
the complete relevant Android CoreCLR API30 batch passed **269/273 with
0 failures and 4 existing skips**; the complete native Windows batch
passed **216/225 with 0 failures and 9 existing skips**. All **21
Android and 13 Windows Flyout fixture cases passed**, including existing
screenshot assertions. Both complete case-name inventories match the
original published batches. Both relevant batches passed on their first
attempt; no retry occurred.

| Back case | Android run 46162268 | Windows run 46161974 |
|---|---|---|
| Event raised | [Passed, result
100134](https://dev.azure.com/dnceng-public/public/_apis/test/Runs/46162268/results/100134)
| [Passed, result
100062](https://dev.azure.com/dnceng-public/public/_apis/test/Runs/46161974/results/100062)
|
| Handled false permits navigation | [Passed, result
100072](https://dev.azure.com/dnceng-public/public/_apis/test/Runs/46162268/results/100072)
| [Passed, result
100196](https://dev.azure.com/dnceng-public/public/_apis/test/Runs/46161974/results/100196)
|
| Handled true stops navigation | [Passed, result
100212](https://dev.azure.com/dnceng-public/public/_apis/test/Runs/46162268/results/100212)
| [Passed, result
100141](https://dev.azure.com/dnceng-public/public/_apis/test/Runs/46161974/results/100141)
|

Raw native logs 5466 (Android) and 5143 (Windows), full paginated
results, and both downloaded TRXs were inspected. Local pinned
SDK11.0.100 / Android37.2.29 CoreCLR qualification also passed all 21
existing Android Flyout cases on an owned API30 emulator; both managed
clients and the Android HostApp compiled successfully. No physical Apple
or local macOS XCTest automation was used.

The broader default pipeline is still running and has other batches in
failed/running states. This is qualification of the narrow Flyout
fixture, **not a claim that all CI is green**. The original source
passed this back case first and failed it on retry, so these results are
not represented as a deterministic baseline-red/fix-green/revert-red
causal certification.

Upstream recheck found no duplicate back-fixture implementation: #39779
currently changes no Flyout files; #38822 touches this test file only
for unrelated RTL/title setup changes.

### Issues Fixed

<!-- Please make sure that there is a bug logged for the issue being
fixed. The bug should describe the problem and how to reproduce it. -->

Follow-up to #39742: corrects the newly observed Flyout handled-true
fixture race in [build
1658120](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1658120),
Android retry run 46159644 / result 100203 (`NullReferenceException`,
4853 ms, old test line 512). Original first/retry logs, failure
accessibility bytes, and native TRXs are preserved with verified SHA256
hashes. The two separate Bugzilla55912 first-attempt failures were
initial GridLabel/StackLabel waits before any tap and are not modified
by this PR.

<!--
Are you targeting main? All PRs should target the main branch unless
otherwise noted.
-->

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1b5d83ad-4b4b-4107-9a42-c1ada4de4e38

This branch was previously deployed

1 inactive (outdated) deployment
copilot-pat-pool — fbbdd444 Deployed Sep 23, 2026 by kubaflo via conclusion #2047
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-testing Unit tests, device tests platform/android platform/ios platform/macos macOS / Mac Catalyst

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants