Skip to content

Enforce 2s minimum gap between port disable and re-enable in enable_only - #15032

Merged
Nir-Az merged 2 commits into
realsenseai:developmentfrom
Nir-Az:recycle-gap
May 7, 2026
Merged

Nir-Az merged 2 commits into
realsenseai:developmentfrom
Nir-Az:recycle-gap

Conversation

@Nir-Az

@Nir-Az Nir-Az commented May 6, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Defensive guard in Acroname.disable_ports / Acroname.enable_ports that enforces a minimum 2 s gap between issuing a port-disable and a subsequent port-enable on the same hub. Below this threshold the Acroname USBHub3p (firmware 2.12.2) BrainStem control endpoint can stop responding under sustained disable/enable traffic on high-current ports — most reliably reproduced with D585S, which idles around 1 A.

This is not a fix for a deterministic failure. The wedge is intermittent in production. The patch removes the timing trigger by guaranteeing a known-safe minimum, so the failure mode cannot be hit even when other layers (test retry path, fast machine, post-streaming cleanup) compress the gap.

Implementation

  • Module-level constant _MIN_PORT_RECYCLE_GAP_S = 2.0 in acroname.py
  • Acroname.__init__ adds _last_disable_t = None
  • Acroname.disable_ports records time.monotonic() after a successful disable
  • Acroname.enable_ports checks at start and sleeps just enough to honour the gap; logs log.d(f"waiting {settle:.2f}s for Acroname port-recycle gap")
  • Removed two redundant import time shadows that would have caused UnboundLocalError when the module-level time is referenced earlier in the function

devices.py is untouched — initial revision had the logic there but per review (@OhadMeir) it belongs in the hub-specific class.

Threshold data (vtglnx163, D585S on Acroname port 4)

Inter-cycle gap Result
1.0 s (×10) Wedge: BrainStem control endpoint dies after ~8 cycles, full -71 cascade across both stems. Recovery: physical DC power-cycle of the hub.
1.5 s (×10) Hub stays alive, but D585S enumeration errors (device not accepting address ... -62); only 3/10 cycles complete USB enumeration
2.0 s (×30) Clean: 0 failures, no dmesg errors, full re-enumeration each cycle
3.0 s / 5.0 s Clean

libci 11369 showed disable→enable gaps as low as 259 ms when consecutive tests target the same device.

Validation on real pytest workload

Ran pytest unit-tests/live/ --device D585S --context nightly against the same suite, with and without the patch:

Metric With patch Without patch
Passed / Failed / Errors / Skipped 29 / 2 / 2 / 115 29 / 2 / 2 / 115
D585S re-enumerations 23 30
err = -71 events 0 0
cannot disable events 0 0
Hub wedge No No
Duration 358 s 334 s

Same exact failure list both times — the 4 non-passes are test-internal (pytest-enumerate-devices, pytest-rs-convert-to-db3 FAILED; multi_devices/* ERROR) and not caused by hub timing. Cost of the patch on this 6-minute run: ~24 s (~7%).

Today we couldn't reliably trigger the wedge through pytest's natural pacing (re-enumeration takes ~3.8 s, naturally giving the hub breathing room), but yesterday's stress tests prove the trigger is real once cycle gaps drop below 2 s.

Test plan

  • Local syntax check (py_compile)
  • Stress test on real hardware (vtglnx163, D585S): 30× rapid cycles at 2 s gap — clean
  • Real pytest workload (--device D585S --context nightly) with and without patch — same results
  • Verified pre-fix state still wedges at 1 s gap, fails enumeration at 1.5 s gap
  • libci nightly run end-to-end on shared rigs

🤖 Generated with Claude Code

@Nir-Az
Nir-Az requested a review from OhadMeir May 6, 2026 13:28

@Nir-Az Nir-Az left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review — PR #15032 · Enforce 2s minimum gap in enable_only

The fix is correct and well-motivated — the empirical data table in the PR description is excellent. A few items to address before merging:

Summary of inline comments

Line Severity Topic
620 ⚠️ Bug disable_t set after hub.disable_ports() — should be before to cover async port-off
620 ⚠️ Enhancement Use timestamp() (perf_counter) instead of time.time() — already imported, monotonic
625 ⚠️ Enhancement Magic number 2.0 → named constant _MIN_PORT_RECYCLE_GAP_S
627 💡 Enhancement Add log.d(...) before the sleep for CI visibility

Missing automated test

The test plan is entirely manual / libci-run. Consider adding a unit test (e.g., in unit-tests/py/) that:

  • Mocks hub.disable_ports, _wait_until_removed, and time.sleep
  • Verifies that when _wait_until_removed returns in < 2 s, time.sleep is called with a value ≈ 2.0 − elapsed
  • Verifies that when _wait_until_removed already took ≥ 2 s, time.sleep is not called

This would prevent the gap from being accidentally removed in a future refactor.

🤖 Auto-generated comment by rs-agentic bot

Comment thread unit-tests/py/rspy/devices.py Resolved
Comment thread unit-tests/py/rspy/devices.py Resolved
Comment thread unit-tests/py/rspy/devices.py Resolved
Comment thread unit-tests/py/rspy/devices.py Outdated
hub.disable_ports( enabled_ports )
disable_t = time.time()
_wait_until_removed( sns_to_remove, timeout = timeout )
# Acroname USBHub3p firmware (2.12.2) wedges with -71 cascades when

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rephrase. wedges, cascades, not clear

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rewrote in f3370bf. The new docstring on the constant reads:

Minimum elapsed time (seconds) required between issuing a port disable and a subsequent port enable on the Acroname USBHub3p (firmware 2.12.2). Below this threshold, sustained disable/enable traffic on high-current ports (observed with D585S, ~1 A idle) can leave the BrainStem control endpoint unresponsive.

Comment thread unit-tests/py/rspy/devices.py Resolved
…ports with named constant and monotonic timer
@Nir-Az

Nir-Az commented May 6, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed review comments in f3370bf9d:

Comment Resolution
@OhadMeir: rephrase comment, drop "wedges/cascades" jargon Rewritten — comment now says "can leave the BrainStem control endpoint unresponsive" with no jargon
@OhadMeir: push to hub-specific (Acroname only) Moved entirely out of devices.py. Logic now lives in Acroname.disable_ports (records timestamp) and Acroname.enable_ports (waits if needed). devices.py is back to baseline.
@OhadMeir: Linux-only Kept global per discussion — applies on all OSes inside the Acroname class
Bot: use timestamp() / monotonic Switched to time.monotonic()
Bot: magic number → named constant Added module-level _MIN_PORT_RECYCLE_GAP_S = 2.0
Bot: log before sleep Added log.d(f"waiting {settle:.2f}s for Acroname port-recycle gap")
Bot: capture timer before disable Kept the timestamp at end of disable_ports (after the BrainStem command succeeded) — that's a more accurate "port-off-since" anchor than capturing before the command was sent

Also fixed a latent name-shadowing bug in both functions: the existing import time inside the if-blocks made time a local variable for the entire function, which would have triggered UnboundLocalError on the new top-of-function time.monotonic() call. Removed the redundant inner imports — the module-level import time already covers them.

@Nir-Az
Nir-Az merged commit 76cf77b into realsenseai:development May 7, 2026
30 checks passed
@Nir-Az
Nir-Az deleted the recycle-gap branch May 7, 2026 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants