Repository navigation
[CI] NativeAOT CGImageMetadataTest crashes retaining CGImageMetadataTag proxy #26545
Description
Summary
MonoTouchFixtures.CoreGraphics.CGImageMetadataTest.EnumerateMetadata intermittently crashes in NativeAOT while constructing and retaining a CGImageMetadataTag proxy returned by CGImageMetadataEnumerateTagsUsingBlock.
- Stable signature:
iOS simulator NativeAOT CGImageMetadataTest.EnumerateMetadata crashes in CGImageMetadataTag proxy retain (objc_retain EXC_BAD_ACCESS/SIGSEGV) - Classification: genuine intermittent NativeAOT interop/product crash
- Distinct monitored PRs: 2
- Confirmed occurrences: 3
- First retained occurrence: 2026-09-01
- Latest occurrence: 2026-09-02T18:16:42Z
Confirmed occurrences
PR #26424
Two different heads reproduced the same crash:
- Head
f4d27dbb1d3365e1729bf029041aac72cb8c28cb, build 15118520 attempt 1,T: monotouch_ios. - Head
f405da213809a39a7ae21ec5391237633659b77c, build 15157901 attempt 1,T: monotouch_ios.
A later #26424 head completed its simulator build successfully and the PR merged, consistent with an intermittent failure rather than a deterministic merge regression.
PR #26539
- Head:
a279fec7a28ee8515c20640c2c78c7ec8ad11c28 - Build 15185386 attempt 1,
T: monotouch_ios/ Run tests - Worker:
AcesShared 159 - Configuration: iOS simulator Release, trimmable static registrar, NativeAOT
- Simulator: iPhone 14 / iOS 26.5
- Exception:
EXC_BAD_ACCESS,SIGSEGV, invalid address inobjc_retain - Faulting stack:
objc_retainCFObject.CFRetainNativeObject.RetainCGImageMetadataTagproxyCreateObjectRuntime.ConstructINativeObjectRuntime.GetINativeObjectCGImageMetadata.TagEnumeratorCGImageMetadataEnumerateTagsUsingBlockCGImageMetadataTest.EnumerateMetadata
PR #26539 only changes AppKit enum documentation and documentation-test baselines, which are unrelated to CoreGraphics, ImageIO, or NativeAOT object construction.
Impact and reproduction pattern
The crash terminates the entire NativeAOT test application, loses the rest of that configuration’s test results, and fails monotouch_ios. The same managed test and native proxy-retain stack have occurred on multiple heads and unrelated PRs.
Investigation
The Azure timelines, TestSummary and HTML artifacts, raw .ips crash report, faulting-thread image map, retained occurrence ledger, PR file lists, and later passing build progression were compared. The newest run also contains a separate NativeAOT CTLineTests.EnumerateCaretOffsets Objective-C cache abort; that is a different signature and is not grouped here.
Separate searches across every open and closed issue used the exact test name, CGImageMetadataTag, objc_retain, CGImageMetadataEnumerateTagsUsingBlock, proxy construction, and NativeAOT terms. No matching issue was found.
Current diagnosis and workaround
The native callback supplies a metadata-tag handle that is invalid by the time the generated proxy path calls CFRetain/objc_retain, or its ownership classification is intermittently incorrect. The exact lifetime transition is unresolved.
The current workaround is an evidence-backed simulator-stage retry when the parent run is terminal and no deterministic failure would be rerun. Useful follow-up diagnostics would record the callback-provided handle, ownership flag, native type, proxy creation path, and retain timing before dereferencing it.
Azure logs and artifacts are retention-limited.
Activity
dalexsoto commentedon Sep 2, 2026
Retry update for PR #26539 @
a279fec7a28ee8515c20640c2c78c7ec8ad11c28:T: monotouch_iosproduced theCGImageMetadataTest.EnumerateMetadataproxy-retain crash documented above and a separateCTLineTests.EnumerateCaretOffsetsObjective-C cache-corruption abort.monotouch_ios; the PR has one automatic simulator build and no observable prior retries.simulator_tests(forceRetryAllJobs: false). Azure accepted it and created simulator stage attempt 2;monotouch_iosis currently active.The issue body has no Additional info to collect checklist. The retry outcome will be reported when attempt 2 reaches a terminal state.
dalexsoto commentedon Sep 2, 2026
Retry outcome for PR #26539 @
a279fec7a28ee8515c20640c2c78c7ec8ad11c28:T: monotouch_iosattempt 2 succeeded onAcesShared 2, running from2026-09-02T19:05:08Zto21:06:22Z.21:06:31Z; build 15185386 completed successfully at21:10:21Z.CGImageMetadataTest.EnumerateMetadataproxy-retain crash nor the separateCTLineTests.EnumerateCaretOffsetsObjective-C cache abort recurred.21:11:26Z.The canonical count remains 3 confirmed CGImageMetadata occurrences across 2 PRs. The issue body has no Additional info to collect checklist.
dalexsoto commentedon Sep 4, 2026
New occurrence and retry update for PR #26561 @
626d112132dd1e8962770958258776cb20c3c0a9:T: monotouch_ios/Run testscrashed in iOS simulator Release NativeAOT onAcesShared 65.EXC_BAD_ACCESS/SIGSEGV/KERN_INVALID_ADDRESS; termination signal 11.objc_retainCFObject.CFRetainNativeObject.RetainCGImageMetadataTagproxyCreateObjectRuntime.ConstructINativeObjectRuntime.GetINativeObjectCGImageMetadata.TagEnumeratorCGImageMetadataTest.EnumerateMetadataThis raises the canonical count to 4 confirmed occurrences across 3 distinct monitored PRs (#26424, #26539, and #26561).
The unchanged head is conflict-free, the run is terminal, no equivalent retry existed, and the PR had no observable prior retry. The monitor requested the Azure DevOps equivalent of Rerun failed jobs for
simulator_tests(forceRetryAllJobs: false). Azure accepted it and created attempt 2 with onlymonotouch_iospending. Retry usage is now 1/25 PR lifetime and 1/10 for this head.Issue-specific checklist: none; the issue body has no Additional info to collect section.
dalexsoto commentedon Sep 4, 2026
Retry outcome for PR #26561 @
626d112132dd1e8962770958258776cb20c3c0a9:T: monotouch_iosjob.monotouch_iossucceeded from2026-09-04T12:39:23Zto14:40:14Z.14:40:24Z.14:45:15Z.succeededWithIssuesrecords.CGImageMetadataTest.EnumerateMetadataobjc_retaincrash did not recur.The canonical count remains 4 confirmed occurrences across 3 monitored PRs.
Issue-specific checklist: none; the issue body has no Additional info to collect section.
dalexsoto commentedon Sep 4, 2026
PR #26561 merged at
2026-09-04T15:37:20Zafter itsmonotouch_iosretry and simulator stage completed successfully.The
CGImageMetadataTest.EnumerateMetadataNativeAOT crash did not recur on attempt 2. The canonical count remains 4 confirmed occurrences across 3 monitored PRs, and retry accounting for #26561 remains 1/25 lifetime and 1/10 for head626d112132dd1e8962770958258776cb20c3c0a9.Issue-specific checklist: none; the issue body has no Additional info to collect section.
dalexsoto commentedon Sep 8, 2026
CI recurrences and retry outcome: PR #24710
1b2afec17a6d00607843a89f66efa21add10eef3T: monotouch_ios→Run testscrashed in iOS simulatorRelease (trimmable static registrar, NativeAOT).EXC_BAD_ACCESS/SIGSEGV/KERN_INVALID_ADDRESSon the main thread:objc_retain → CFObject.CFRetain → NativeObject.Retain → CGImageMetadataTag proxy CreateObject → CGImageMetadata.EnumerateTags → CGImageMetadataTest.EnumerateMetadata.simulator_testsjob. Stage attempt 2 —T: monotouch_ios→Run testscompleted failed after reproducing the exact same configuration, exception, and proxy-retain stack. No additional retry was requested.Additional info to collectsection.dalexsoto commentedon Sep 9, 2026
Retry outcome: PR #24710
1b2afec17a6d00607843a89f66efa21add10eef3.T: monotouch_ios/Run testscompleted successfully. The job ran from2026-09-09T15:24:28Zto17:39:42Z; the stage completed at17:39:49Z.CGImageMetadataTest.EnumerateMetadataNativeAOTobjc_retain → CFObject.CFRetain → NativeObject.Retaincrash did not recur.Additional info to collectsection.dalexsoto commentedon Sep 22, 2026
Confirmed NativeAOT recurrence: PR #26669
5f6fafc37b48f2ccf45cc3a0f234500ba668bebc.xamarin-macios-sim-pr-tests(21922), 15410988 / monotouch_ios / Run tests, stage/job/task attempt 1, task log 2890.net11.0-ios,iossimulator-arm64, Release / trimmable-static registrar / PublishAot=true.The app, PID 44202, connected at 2026-09-22 08:13:24 UTC and crashed at 08:14:32 UTC with EXC_BAD_ACCESS / SIGSEGV / KERN_INVALID_ADDRESS, termination signal 11. Faulting thread 0 independently resolves to the exact established path:
Attribution: the incident was collected during the next successful configuration, execution132. Its PID/time match the earlier failed NativeAOT execution130, not execution132's later PID45093. The later configuration's passing NUnit results are not assigned to the crashed app.
mlaunchreturned0, but the app terminated by signal11 and the harness explicitly reported a crash without results.Classification/counts: the same genuinely intermittent NativeAOT interop crash; the exact ownership/lifetime transition remains unresolved. The durable mapping was reconciled with the six previously documented occurrences across four PRs in this issue. Those six were adopted as already-reported history, not fresh Azure observations. Adding this one new PR/occurrence gives 7 confirmed occurrences across 5 distinct monitored PRs.
Retry decision: no retry submitted. This run also has a deterministic app-size baseline failure and other work remained active; no source change is authorized on this bot branch.
Diagnostics/checklist: the refreshed body has no Additional info to collect section. Exact head/run/attempt/job/task, executable properties, app PID/timing and the original symbolicated incident were collected. Native app UUID:
4c88e5cb-12aa-3db5-a567-3d05db13e4d3; the incident reports host macOS26.6.2/25G83 and the run targets an iOS27.0 simulator. No separate managed exception or completed failed-app NUnit log/results are available. The precise lifetime cause and unavailable historical task/timestamp details are not invented. Worker identifiers and IPs are withheld.Evidence:
HtmlReport-simulator_testsmonotouch_ios-1,tests/monotouch-test/130/plustests/DiagnosticReports/monotouchtest-2026-09-22-011520.ipsin the linked run; artifacts are retention-limited.dalexsoto commentedon Sep 23, 2026
Confirmed NativeAOT recurrence: PR #26574, new XHarness head
306b9621faa1e9c5b6da750f2d701ceef2c2d378.xamarin-macios-sim-pr-tests(21922), 15425071 / monotouch_ios / Run tests, stage/job/task attempt1, task log 2927.net11.0-ios,iossimulator-arm64, Release,PublishAot=true, trimmable-static registrar.App PID 45284 connected at 2026-09-23 03:58:15 UTC and crashed at 03:59:53 UTC with EXC_BAD_ACCESS / SIGSEGV / KERN_INVALID_ADDRESS, termination signal11. Faulting thread0 independently resolves to:
The incident was collected during later execution130, but its PID and capture time match failed execution128. Later configuration results are not assigned to the crashed application. The launcher returned0; the harness nevertheless recorded an app crash without results, consistent with the native incident.
Classification/counts: the established genuinely intermittent NativeAOT proxy-retain crash. This new PR/head/run raises the canonical total to 8 confirmed occurrences across6 distinct monitored PRs. The precise ownership/lifetime transition is not inferred from this incident alone.
Fix/retry context: linked fix PR #26676 remains open and unmerged. No claim that this build contains that fix or invalidates it is made. No retry was submitted: this PR also has a deterministic size failure and a source-related package-build failure, and all retry-accounting gates remain required.
Diagnostics/checklist: the freshly read issue body has no Additional info to collect section. Exact head/run/attempt/job/task, app PID/timing, executable properties and the original symbolicated incident were collected. Completed failed-app NUnit results/TestLog and any separate managed exception are unavailable; no fix/recovery is claimed. Worker identifiers/IPs are withheld.
Evidence:
HtmlReport-simulator_testsmonotouch_ios-1,tests/monotouch-test/128/andtests/DiagnosticReports/monotouchtest-2026-09-22-210036.ipsin the linked run. Artifacts are retention-limited. No source or branch changes were performed.dalexsoto commentedon Sep 23, 2026
Exact NativeAOT recovery on an external retry; separate failures remain
PR #26574, unchanged head
306b9621faa1e9c5b6da750f2d701ceef2c2d378, reran 15425071 / monotouch_ios / Run tests, pipelinexamarin-macios-sim-pr-tests(21922), stage/job attempt 2, task-record attempt 1, log 3170.Original failed execution 128, Release / NativeAOT / trimmable-static registrar, genuinely completed on this attempt. Its original XML records 3,698 passed, 191 skipped, 30 inconclusive, zero failed, including:
Executable-scoped properties match the original failure:
net11.0-ios,iossimulator-arm64,Configuration=Release,PublishAot=true,Registrar=trimmable-static,UseMonoRuntime=false. The listener connected andmlaunchexited 0; that launcher status is not substituted for a separate app exit.Not whole-job or whole-PR recovery: a different configuration, Debug (static registrar), execution122, connected and then exceeded the harness's 900-second deadline. Its real TestLog, active test, app stack and separate exit are unavailable; LLDB sampled the launcher, not the app. That hang remains unresolved and is not added to this crash signature. The independent macOS size mismatch also persisted. The pipeline ended failed at 2026-09-23 14:23:55 UTC.
Counts remain 8 confirmed crash occurrences across 6 distinct monitored PRs. Recovery creates no new failure and does not establish a source fix or resolve the original ownership/lifetime question. The issue remains open. This was an externally initiated retry; no further monitor retry or source change was made, and exact all-actor accepted-request accounting remains uncertified.
Diagnostics/checklist: the refreshed issue body has no Additional info to collect section. Correct-attempt original NUnit leaves and exact case, executable properties, connection/launcher status and terminal job/run evidence were collected. Original failed-app results/TestLog and precise lifetime cause remain unavailable; passing-attempt data does not retroactively supply them. Worker identifiers and IPs are withheld.
Evidence:
HtmlReport-simulator_testsmonotouch_ios-2, originaltests/monotouch-test/128/test-ios-20260923_062610.xmland executable build127 diagnostic log. Azure artifacts are retention-limited.dalexsoto commentedon Sep 23, 2026
Independently confirmed NativeAOT recurrence: PR #26677
c68e18446410437640b16187ba4b797ccc88b4c6.net11.0-ios,iossimulator-arm64,PublishAot=true,Registrar=trimmable-static.App PID 46587 connected at 2026-09-23 16:17:04 UTC, then crashed at 16:18:51.6326 UTC with EXC_BAD_ACCESS / SIGSEGV, termination signal11. Faulting thread0 independently resolves to:
Attribution: the incident also appears under later execution132, but its PID and timestamp match failed execution130. Execution132 launched a different PID, 47797; its results are not attributed to this crash.
mlaunchexited0, but the app incident and harness independently establish termination by signal11 without completed results.Classification/counts: the established genuinely intermittent NativeAOT interop crash; the exact ownership/lifetime transition remains unresolved. This new PR/occurrence brings the reconciled total to 9 confirmed crashes across 7 distinct monitored PRs. Duplicate incident copies are not extra occurrences.
Linked fix PR #26676 was refreshed and remains open/unmerged; no claim that this build contains or invalidates that fix is made. The iOS job ended failed at 17:03:13 UTC; publication was still in progress at observation. Its separate SpringBoard launch timeout belongs in #26227, and the same run also has the source-level compiler/copy race in #26688.
No monitor retry or Rolf-branch source/worktree/commit/push change was performed. Exact all-actor retry accounting remains uncertified.
Diagnostics/checklist: the freshly read body has no Additional info to collect section. Original run logs, exact executable properties, app PID/timing and symbolicated incident were collected. Completed failed-app NUnit results/TestLog, any separate managed exception, and the precise lifetime cause remain unavailable. Worker identifiers and IPs are withheld.
Evidence:
HtmlReport-simulator_testsmonotouch_ios-1, execution130, and globaltests/DiagnosticReports/monotouchtest-2026-09-23-091942.ips. Azure artifacts are retention-limited. The canonical issue remains open.AI-assisted CI monitoring update.
dalexsoto commentedon Sep 23, 2026
Confirmed NativeAOT recurrence: PR #26574, replacement head
fe3c25fda59f9b5966f81fda854967bc5916300c.PublishAot=true,net11.0-ios,iossimulator-arm64,UseMonoRuntime=false.App PID 45185 connected at 2026-09-23 17:02:12 UTC and crashed at 17:03:50.1387 UTC with EXC_BAD_ACCESS / SIGSEGV, termination signal11. Faulting thread0 independently resolves to:
Attribution uses the failed app's PID and capture time, not the directory where a copy of the incident was collected. The launcher returned0, but the app incident and harness explicitly establish a crash without completed results.
Classification/counts: the established genuinely intermittent NativeAOT metadata-tag retain failure. This new head/run adds one occurrence on an already counted PR: 10 confirmed crashes across 7 distinct monitored PRs. The earlier same-PR NativeAOT retry recovery remains historical evidence, not proof that later heads cannot reproduce the crash.
Linked fix PR #26676 remains open/unmerged at this investigation; no post-fix regression claim is made. The pipeline ended failed at 17:56:08 UTC, also retaining the separately tracked source copy race and linker launch failure. The macOS expected-size fix passed independently.
No monitor retry or bot-branch source change was performed. The source race and uncertified all-actor retry accounting continue to block additional monitor retries.
Diagnostics/checklist: the freshly read body has no Additional info to collect section. Exact run/head/attempt/job/task/configuration, app connection timing, native PID/stack and termination were collected. Completed failed-app NUnit results/TestLog, a separate managed exception and the exact ownership/lifetime transition remain unavailable. Worker identifiers and IP addresses are withheld.
Evidence:
HtmlReport-simulator_testsmonotouch_ios-1, execution128 and its original app incident in the linked run. Artifacts are retention-limited. The issue remains open.AI-assisted CI monitoring update.
3 remaining items
dalexsoto commentedon Sep 24, 2026
Linked fix PR #26676: exact NativeAOT regression tests passed
On #26676, head
967512dd451da1a2df9140c6d786ecdd26359db6, Azure run 15431425, attempt 1,T: monotouch_iosjoba70ed34c-97c9-5ebb-c5be-8e2604f5ffccsucceeded. InHtmlReport-simulator_testsmonotouch_ios-1,tests/monotouch-test/130/vsts-nunit-test-ios-20260923_091459.xmlrecords Passed forMonoTouchFixtures.CoreGraphics.CGImageMetadataTest.EnumerateMetadataand bothEnumerateMetadataWithForcedGC(False/True)cases. The report identifies execution 130 as Release (trimmable static registrar, NativeAOT), the configuration of this issue's intermittentEXC_BAD_ACCESS/SIGSEGVproxy-retain crash (objc_retain/CFRetain/CGImageMetadataTag).The same three cases also passed in Release (NativeAOT) execution 128 and Release (NativeAOT, .NET 11 defaults) execution 158. The iOS TestSummary reports all 25 executed variations passed. This verifies execution and a successful result on the proposed fix head; it does not establish that the intermittent crash is eliminated. The fix PR is still open and unmerged. Separately, the externally initiated retry on the unfixed #26677, head
c68e18446410437640b16187ba4b797ccc88b4c6, Azure run 15431402 simulator attempt 3 remains in progress, so there is no new failure or recovery outcome to report from that retry. The monitor requested no retry or source edit.Existing failure totals remain 13 confirmed occurrences across 8 PRs. This positive fix-branch observation is not another failure occurrence. The issue body contains no
Additional info to collectchecklist; no Helix work item applies to this Azure simulator job. Azure report artifacts are retention-limited.AI-assisted CI monitoring update.
dalexsoto commentedon Sep 24, 2026
Exact NativeAOT metadata proxy-retain crash recurred on #26677 attempt 3
#26677, unchanged head
c68e18446410437640b16187ba4b797ccc88b4c6: Azure run 15431402 / simulator stage attempt 3 /T: monotouch_iosjobf184b60d-41ed-595a-0d93-d653d9488b65,Run teststask5c7afd39-9bd2-5231-57bf-428e88fb17a1, failed in Release (NativeAOT) execution 128. The failed app was PID 45273; its native incident at 01:25:41 PDT showsEXC_BAD_ACCESS/SIGSEGVinobjc_retain→CFObject.CFRetain→CGImageMetadataTag.Proxy.CreateObject→CGImageMetadataTest.EnumerateMetadata. The same PID/fixture appears in the execution's stderr, and this job's crash-upload log 3432 published the incident. Evidence:HtmlReport-simulator_testsmonotouch_ios-3,tests/monotouch-test/128/monotouchtest-2026-09-24-012546-20260924_012547.ips. There is no completed NUnit XML for the crashed execution.This is a new confirmed attempt/job/test occurrence of the existing
crash|iOS-simulator|trimmable-static-registrar-NativeAOT|MonoTouchFixtures.CoreGraphics.CGImageMetadataTest.EnumerateMetadata|EXC_BAD_ACCESS-SIGSEGV|CFRetainsignature, classified as a genuinely intermittent NativeAOT product crash, not a new distinct PR. The previous failed attempts remain separately recorded. Simulator stage attempt 3 and the parent build both ended failed. Proposed fix #26676 is open and unmerged; this unfixed head does not test or invalidate that fix.Totals: 14 confirmed failure occurrences across 8 distinct monitored PRs. The retry was externally initiated; its accepted-request actor/grouping and all-head counts remain uncertified, and the monitor requested no further retry or source edit. The issue body has no
Additional info to collectsection. No Helix work item applies to this Azure simulator job; artifacts are retention-limited.AI-assisted CI monitoring update.
dalexsoto commentedon Sep 24, 2026
Exact NativeAOT metadata crash on a new monitored PR: #26687
#26687, head
6df0ea155215603133a01b583c94440eeb32ca07, Azure run 15431340 / simulator stage attempt 2 /T: monotouch_iosjoba82441db-bb83-5a9b-9451-a8501e963a31,Run teststask2ab0f981-bb57-5e5c-6794-d1fea837463a: Release (trimmable static registrar, NativeAOT) execution 130 crashed. The failing app, PID 46717, reachedCGImageMetadataTest.EnumerateMetadata, then faulted at 11:36:31 PDT withEXC_BAD_ACCESS/SIGSEGVinobjc_retain→CFObject.CFRetain→CGImageMetadataTag.Proxy.CreateObject. This is the exactcrash|iOS-simulator|trimmable-static-registrar-NativeAOT|MonoTouchFixtures.CoreGraphics.CGImageMetadataTest.EnumerateMetadata|EXC_BAD_ACCESS-SIGSEGV|CFRetainsignature.The native incident was collected under later passing execution 132 in
HtmlReport-simulator_testsmonotouch_ios-2, but it belongs to failed execution 130: its PID/time match execution 130's stderr; execution 132 started later and used PID 47863. The failed job's crash-upload log 3226 publishedmonotouchtest-2026-09-23-113712.ips. No NUnit XML was completed for the crashed app. Externally initiated simulator stage attempt 4 later passed all 25 variations on the same head and the parent build succeeded; this is evidence of intermittency, not proof that the crash was fixed. The proposed ImageIO fix PR #26676 remains open/unmerged and was not tested by this unrelated source-fix head.Totals: 15 confirmed failures across 9 distinct monitored PRs. This adds one failure and one PR to the canonical record. The external accepted-request actor/grouping and all-head retry totals remain uncertified; no monitor retry or source edit was made. The issue body has no
Additional info to collectsection. No Helix work item applies to this Azure simulator job; artifacts are retention-limited.AI-assisted CI monitoring update.
[ImageIO] Keep metadata alive while enumerating tags. Fixes #26545 (#…
dalexsoto commentedon Sep 24, 2026
Exact NativeAOT case passed on a fourth external attempt
#26677, unchanged head
c68e18446410437640b16187ba4b797ccc88b4c6: in Azure run 15431402 / simulator stage attempt 4 /T: monotouch_iosjob9f96c276-e12b-58ea-c2fe-da8753b74b32,MonoTouchFixtures.CoreGraphics.CGImageMetadataTest.EnumerateMetadatapassed in NativeAOT execution 128, as recorded inHtmlReport-simulator_testsmonotouch_ios-4attests/monotouch-test/128/test-ios-20260924_060037.xml. The same case had the reported proxy-retain SIGSEGV on attempts 1–3. No additional occurrence is counted; this issue remains at 15 confirmed failures across 9 distinct monitored PRs.The fourth attempt's
Run teststask8f4eca1a-39a6-57b8-2bf4-e54eb13fc92b(log 3557) and parent run failed for a different, unresolved iOS 27.0 Release test-suite hang: execution 156 connected, logged 12 real passing cases, then exceeded 900 seconds without a final NUnit result. A nearby SpringBoard abort preceded the successful app connection and test execution, so it is not evidence that this hang shares the launch-failure root. Do not count the parent failure as another NativeAOT crash or call the PR green.Not reopening: #26545 is closed/completed following merged fix #26676. This retry of an older, pre-fix run passed the exact case but cannot validate the merged fix. The actor and exact accepted-request count remain uncertified; the monitor requested no retry. The issue body has no Additional info to collect section, and there is no Helix work item for this Azure simulator job. HtmlReport artifacts are retention-limited.
AI-assisted CI monitoring update.
dalexsoto commentedon Sep 24, 2026
Fifth external iOS attempt completed successfully; #26545 remains closed
#26677, unchanged head
c68e18446410437640b16187ba4b797ccc88b4c6: Azure run 15431402, Simulator tests stage attempt 5,T: monotouch_ios/Run tests(task log 3701), finished succeeded at 2026-09-24 18:49:00 UTC.TestSummary-simulator_testsmonotouch_ios-5reports all 25 test configurations passed.The exact
MonoTouchFixtures.CoreGraphics.CGImageMetadataTest.EnumerateMetadatacase passed in both Release (NativeAOT) execution 128 (tests/monotouch-test/128/test-ios-20260924_110221.xml) and Release (trimmable static registrar, NativeAOT) execution 130 (tests/monotouch-test/130/test-ios-20260924_110523.xml) withinHtmlReport-simulator_testsmonotouch_ios-5. Each completed 3,698 NUnit tests with zero failures, and the launcher exited 0. Attempts 1–3 crashed in this issue's retain stack; attempt 4 had already passed the exact NativeAOT case but failed a different connected test-suite timeout. That previously hung execution 156 also completed on attempt 5 (3,722 passed, zero failed); it is not another metadata crash.Signature/classification:
crash|iOS-simulator|trimmable-static-registrar-NativeAOT|MonoTouchFixtures.CoreGraphics.CGImageMetadataTest.EnumerateMetadata|EXC_BAD_ACCESS-SIGSEGV|CFRetain/ genuinely intermittent product crash. Counts remain 15 confirmed failure occurrences across 9 distinct monitored PRs; this success is not a new failure. Azure's parent run finished partiallySucceeded at 18:53:49 UTC, retaining historical failed attempts, while the final simulator and publication stages succeeded. GitHub shows 52 successful and 3 skipped checks on this head, and the PR merged at 18:54:13 UTC. A pass on this unchanged PR head does not establish that the merged fix in #26676 eliminated the intermittent crash.Not reopening: #26545 closed/completed today following #26676; a successful retry does not invalidate that closure. The retry was external; its actor, accepted-request grouping, and exact PR/head retry totals remain unavailable, so this monitor submitted no retry. The issue body has no Additional info to collect section. There is no Helix work item for this Azure simulator job; HtmlReport artifacts are retention-limited.
dalexsoto commentedon Sep 25, 2026
Exact NativeAOT metadata-tag proxy crash on release PR #26703
f23610d3944070f9d46892bd45a3fe544d5a8013;release/11.0.1xx-rc.2target.T: monotouch_ios/Run tests(pipeline 21922, stage/job attempt 1, task log 2853);monotouch-test / iOS 27.0 simulator / Release (NativeAOT), execution 128. Correct-attempt TestSummary reports Crashed; the app connected before its PID 43016 sufferedEXC_BAD_ACCESS/SIGSEGVat 2026-09-24T13:09:57.859300Z.objc_retain -> CFObject.CFRetain -> NativeObject.Retain -> CGImageMetadataTag.Proxy.CreateObject -> CGImageMetadataTest.EnumerateMetadataintests/monotouch-test/128/monotouchtest-2026-09-24-061001-20260924_061003.ips. This is the established [CI] NativeAOT CGImageMetadataTest crashes retaining CGImageMetadataTag proxy #26545 app crash, not the separate SpringBoard launch failure in execution 102. Counts: 16 confirmed occurrences across 10 monitored PRs.Issue-specific diagnostics: The refreshed issue body has no Additional info to collect section. PR/head/run/attempt/job/task, NativeAOT configuration, app PID, incident time and faulting stack were collected; completed NUnit XML for the crashed execution and a Helix work item are unavailable. Worker identities/IPs are withheld; artifacts have limited retention.
State/action: #26545 remains CLOSED / COMPLETED by #26676, closed 2026-09-24. This release-branch run is not verified to include that recent main-branch fix; it does not invalidate the closure. Do not reopen. No monitor retry or Rolf source change; the run also has a deterministic AppSize failure, and all-actor retry counts are uncertified.
AI-assisted CI monitoring update.
dalexsoto commentedon Sep 25, 2026
Exact NativeAOT ImageIO proxy-retain crash on release PR #26709
#26709, head
ea9aa7c1d52bec51507dfbe7b58777fe328042f2, failed Azure simulator run 15463830 /T: monotouch_ios/Run tests(pipeline 21922, stage/job/task attempt 1, task log 2890). The iOS 27 simulator Release (NativeAOT) app in HtmlReport execution 128, PID 45106, logged entry intoCGImageMetadataTestat 2026-09-25T18:21:05.552928Z and crashed at 2026-09-25T18:21:05.5564Z withEXC_BAD_ACCESS/SIGSEGV. The faulting native stack isobjc_retain→CFObject.CFRetain→CGImageMetadataTagproxy creation →CGImageMetadataTest.EnumerateMetadata, the exact #26545 lifetime/retain signature. The harness recorded that execution as crashed with no NUnit XML.Artifact-timing correction: The native
.ipswas reported at 2026-09-25T18:21:53Z, after execution 128 ended, so HtmlReport placed it under later execution 130. That later run had a different app PID 46264 and passed 3698 NUnit methods, includingCGImageMetadataTest.EnumerateMetadata. Its success is not recovery of the earlier crashing PID. The contemporaneous no-listener SpringBoard timeout in execution136 is separately tracked in #26227.Classification: intermittent source-related ImageIO binding/lifetime crash. Counts: 17 confirmed occurrences across 11 distinct monitored PRs; #26709 adds one new PR/occurrence. This issue was CLOSED / COMPLETED by #26676's main fix on 2026-09-24, but this
release/11.0.1xx-rc.2head has not been verified to include it; the release backport #26713 is still open. Do not reopen the under-two-week-old main issue based on a pre-backport release failure. No monitor retry: source-related root and a second failed configuration in the same job; no edit to Rolf's branch.Issue-specific diagnostics: The refreshed body has no Additional info to collect section. Exact PR/head/run/attempt/job, crashed PID and captured native retain stack, correct execution attribution, and later passing PID/XML were verified from Azure TestSummary, run logs, app stderr and native incident. Failed execution128 produced no NUnit XML; callback handle ownership/proxy instrumentation and a Helix item are unavailable. Azure artifacts are retention-limited; no worker identifiers, IPs or credentials are included.
AI-assisted CI monitoring update.
dalexsoto commentedon Sep 26, 2026
New distinct release PR: #26712 iOS NativeAOT metadata-tag retain SIGSEGV
#26712, head
cdfccd468dcba92b01a0d84c742209ee19bea9d6: the external simulator-stage/T: monotouch_iosattempt 2 in Azure run 15463847 /Run tests(definition 21922, joba82441db-bb83-5a9b-9451-a8501e963a31, task2ab0f981-bb57-5e5c-6794-d1fea837463a, task record attempt 1, log 3170) reported 1 crashed / 24 passed iOS configurations. The crashed variation was Release (trimmable static registrar, NativeAOT), execution 130. App PID 46848 connected to the test listener at 2026-09-26 21:42:10 UTC, logged entry intoCGImageMetadataTestat 21:43:05 UTC, then the native report captured EXC_BAD_ACCESS / SIGSEGV (signal 11) at 21:43:05.3798 UTC. The faulting thread isobjc_retain→CFObject.CFRetain→NativeObject.Retain→CGImageMetadataTag_Proxy.CreateObject→CGImageMetadata.TagEnumerator→CGImageMetadataTest.EnumerateMetadata: the exact #26545 proxy-retain signature.Artifact attribution matters: the delayed
.ipsfilename/report time was 21:43:42 UTC, soHtmlReport-simulator_testsmonotouch_ios-2collected it under later execution 132. But the incident's capture time and PID match failed execution 130, whose stderr ends atCGImageMetadataTestwith no completed NUnit XML. Execution 132 started later with different PID 47692 and ran 3,728 passing / 0 failing cases; it is not the crashing app. The native incident has a resolved stack but no module version strings; those, the crashed execution's final NUnit XML, and a Helix work item are unavailable (this is direct Azure CI).Classification: recurring intermittent NativeAOT product/interop crash, not this PR's
msbuild/ILMerge.targetschange. #26712 adds one newly confirmed occurrence on a new monitored PR: 18 failures across 12 distinct PRs for canonical #26545. Its two previously stalled NSUrlSession redirect executions passed their exact three subcases on attempt 2, but the iOS job and parent run are still red/in progress for this separate crash; the Mac Catalyst output-race path is separately tracked. No monitor retry or Rolf source edit occurred; exact all-actor PR/head retry counts are uncertified.#26545 remains CLOSED / COMPLETED by recent main fix #26676 (closed 2026-09-24 10:31:39 UTC). This
release/11.0.1xx-rc.2head has not been verified to contain that fix and does not justify reopening an under-two-week closure. The refreshed issue body has no Additional info to collect checklist. Azure artifacts are retention-limited; no worker identifier or IP is disclosed.AI-assisted CI monitoring update.
dalexsoto commentedon Sep 26, 2026
#26712 external iOS attempt 2 finished red from the reported proxy-retain crash
The new #26712 NativeAOT execution-130 SIGSEGV on head
cdfccd468dcba92b01a0d84c742209ee19bea9d6was already reported with its PID-matchedobjc_retain→CFObject.CFRetain→CGImageMetadataTag_Proxy.CreateObjectstack. New since that report: Azure simulator run 15463847 finished failed at 2026-09-26 22:25:20 UTC after its externally reexecutedSimulator testsstage attempt 2 finished failed at 22:21:36 UTC.T: monotouch_iosjob attempt 2 finished failed at 22:21:30 UTC; its first failedRun teststask (task record attempt 1, log 3170) is the same Release (trimmable static registrar, NativeAOT) execution-130 crash. There is no new #26545 failure occurrence beyond the one already reported.The separately rerun Mac Catalyst job passed its old shared-output build-race path (#26688), and the old iOS redirect-stalled executions passed their exact cases (#26221); neither clears this independent SIGSEGV. Canonical counts remain 18 confirmed failures across 12 monitored PRs. Crash execution 130 has no completed NUnit XML; the resolved native stack was in its delayed
.ips(collected under a later passing execution), and there is no Helix work item for this direct Azure job.#26545 remains CLOSED / COMPLETED following the recent main fix #26676; this
release/11.0.1xx-rc.2head is not verified to contain that fix, so not reopening. The issue body has no Additional info to collect section. No monitor retry or Rolf source edit was made; the external actor/payload and exact all-actor retry counts remain uncertified. Azure artifacts are retention-limited.AI-assisted CI monitoring update.
[release/11.0.1xx-rc.2] [ImageIO] Keep metadata alive while enumerati…
dalexsoto commentedon Sep 28, 2026
#26698: exact iOS NativeAOT metadata-tag retain SIGSEGV on a pre-fix-queued run
#26698, head
cac222ec67c6e28a9b8c61d137d018e126385395(rolfbjarne): the externally initiated Simulator tests stage/job attempt 2 in Azure run 15444300 failedT: monotouch_iosjoba82441db-bb83-5a9b-9451-a8501e963a31/Run teststask2ab0f981-bb57-5e5c-6794-d1fea837463a(log 3071).TestSummary-simulator_testsmonotouch_ios-2has 24 passing configurations and oneCrashediOS 27 Release (NativeAOT) execution 128. The app PID 48270 launched and connected, but stopped whileMonoTouchFixtures.CoreGraphics.CGImageMetadataTest.EnumerateMetadatawas running and produced no NUnit results XML.The same-PID native IPS report
monotouchtest-2026-09-24-061219.ipshas internal capture time 2026-09-24 13:11:44 UTC andEXC_BAD_ACCESS/ SIGSEGV (signal 11, segmentation fault) on the faulting thread:objc_retain→CFObject.CFRetain→CGImageMetadataTag_Proxy.CreateObject→CGImageMetadataTest.EnumerateMetadata. This is the exact source-related proxy-retain stackcrash|iOS27-simulator|Release-NativeAOT|CGImageMetadataTest.EnumerateMetadata|EXC_BAD_ACCESS-SIGSEGV|objc_retain-CFRetain, not either of this run's initial iOS redirect-test timeouts (#26221). A third external simulator-stage attempt is now active; no crash recovery is inferred.The canonical ledger now has 19 confirmed occurrences across 13 distinct monitored PRs. This issue is CLOSED/COMPLETED, with closing fix PR #26676 merged 2026-09-24 10:31:38 UTC. The run was queued at 07:10:50 UTC, before that merge; its later attempt-2 crash does not prove the fix was included or has regressed. Do not reopen: the fix-merged closure is less than two weeks old and no failing post-fix main/fix-containing build is established. No monitor retry or Rolf source edit.
Issue-specific diagnostics: The refreshed issue body has no Additional info to collect section. Exact PR/head/run/attempt/job/task, TestSummary, launcher/stderr and symbolicated native PID/time/frames were inspected. No managed exception or NUnit XML survived this NativeAOT crash; no Helix dump/work item exists for this direct Azure job. Fix inclusion is unproven; native artifacts are retention-limited.
dalexsoto commentedon Sep 28, 2026
#26698: exact previously crashing NativeAOT metadata test passed on external attempt 3
The previously reported #26698, unchanged head
cac222ec67c6e28a9b8c61d137d018e126385395(rolfbjarne), Azure simulator run 15444300 stage/job attempt 2T: monotouch_ios/Run tests(log 3071) crashed in iOS 27 Release (NativeAOT) execution 128, with a same-PIDEXC_BAD_ACCESS/ SIGSEGV stackobjc_retain→CFRetain→CGImageMetadataTest.EnumerateMetadata. That historical failure is already counted in this issue.New external retry outcome: Simulator tests stage/job attempt 3
T: monotouch_iosjobf184b60d-41ed-595a-0d93-d653d9488b65/Run teststask5c7afd39-9bd2-5231-57bf-428e88fb17a1(log 3215) reports all 25 iOS configurations passed. The same Release NativeAOT execution 128 contains a valid NUnitresult=Passedfor the exactMonoTouchFixtures.CoreGraphics.CGImageMetadataTest.EnumerateMetadatacase at 2026-09-28 18:19:32 UTC. The job succeeded 19:02:09 UTC, simulator stage 19:02:14 UTC, and parent run completed succeeded at 19:08:19 UTC. Previously stalled redirect configurations 134 and 138 also passed all seven exact subcases each, but belong to #26221, not this crash root.Signature/classification:
crash|iOS27-simulator|Release-NativeAOT|CGImageMetadataTest.EnumerateMetadata|EXC_BAD_ACCESS-SIGSEGV|objc_retain-CFRetain— original source-related metadata-tag proxy-retain crash with a verified same-head external test recovery, 2 confirmed occurrences across 2 distinct monitored PRs for this exact signature; this issue's related signatures total 19 occurrences across 13 PRs. The run originally queued 2026-09-24 07:10:50 UTC, before fix PR #26676 merged around 10:31 UTC and the issue closed COMPLETED at 10:31:39 UTC. A later same-head test pass does not prove the fix was included or erase the original crash. The merged-fix closure is less than two weeks old; do not reopen without a proven post-fix regression.Issue-specific diagnostics: The refreshed issue body has no Additional info to collect section. Exact PR/head/run/attempt/job/task, previous native crash stack, attempt-3 TestSummary, the matching NUnit XML case and terminal stage/parent outcomes were verified; no Helix work item applies to this direct Azure simulator job. No monitor retry or Rolf source edit was made. The external accepted-request payload and complete all-actor 25/PR-lifetime and 10/head counts remain uncertified; Azure artifacts have limited retention.
[src] Keep native callback owners alive during synchronous enumeration (