Skip to content

migrating d400 folder tests to pytest infrastructure - #14965

Merged
Nir-Az merged 34 commits into
realsenseai:developmentfrom
remibettan:migrate_d400_folder_to_pytest
Apr 28, 2026
Merged

Nir-Az merged 34 commits into
realsenseai:developmentfrom
remibettan:migrate_d400_folder_to_pytest

Conversation

@remibettan

@remibettan remibettan commented Apr 23, 2026 •

Copy link
Copy Markdown
Contributor

Tracked by: RSDSO-21356

Overview

Migrates all test files under from the legacy infrastructure to the pytest infrastructure, aligning them with the project standard.

Files Migrated

File Description
Auto-exposure and gain limit tests, including 2-device-instance scenarios
D405 calibration stream tests
Depth auto-exposure metadata verification
Depth AE enable/disable toggle tests
Depth AE convergence rate and failure threshold tests
Depth AE mode (regular/accelerated) tests
Disparity modulation option tests
Emitter frequency support matrix by SKU and FW version
Emitter frequency set/get tests (57 kHz / 91 kHz)
Emitter on/off toggle and streaming tests
Full HDR nightly test suite (12 tests: config, streaming, merge, sequence ID)
HDR sanity streaming test with custom exposure/gain config
MIPI IMU frame index continuity test (Jetson/D457)
Pipeline API test

Infrastructure Fix

  • ****: Fixed a that occurred during pytest collection. Root cause: removes the SDK build directory from after adding it, stripping access to before test modules are collected. Fix: import at conftest module level so it is cached in early — the same pattern already used for .

Code Quality

Applied during migration and in response to code review:

  • ** cleanup guards**: All pipeline / and sensor // calls are wrapped in to ensure cleanup (stop streaming, disable HDR/emitter) happens even when a test assertion or frame-wait raises.
  • Proper negative-test pattern: Replaced bare clauses (which incorrectly catch ) with + for sensor cleanup.
  • ** fix**: The helper in now re-raises the last exception after exhausting retries instead of silently returning .
  • Jetson skip: Tests that require a D455 (, ) are skipped on (CI Jetson) where no D455 is connected.

Copilot AI review requested due to automatic review settings April 23, 2026 07:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Migrates D400 live tests from the legacy rspy.test harness into the repository’s pytest-based infrastructure (RSDSO-21356).

Changes:

  • Replaces multiple unit-tests/live/d400/test-*.py legacy tests with pytest-*.py equivalents using test_device/test_context fixtures and pytest markers.
  • Refactors several existing D400 pytest tests to align with the new infrastructure (assertions, markers, fixtures).
  • Updates unit-tests/conftest.py to import pyrsutils (with a fallback warning) alongside pyrealsense2.

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
unit-tests/live/d400/test-mipi-motion.py Removes legacy rspy.test-based test.
unit-tests/live/d400/test-hdr-sanity.py Removes legacy HDR sanity test.
unit-tests/live/d400/test-hdr-long.py Removes legacy long HDR test suite.
unit-tests/live/d400/test-emitter-frequency.py Removes legacy emitter frequency test.
unit-tests/live/d400/test-emitter-frequency-negative.py Removes legacy negative coverage for emitter frequency support.
unit-tests/live/d400/test-depth_ae_mode.py Removes legacy depth AE mode test.
unit-tests/live/d400/test-depth_ae_convergence.py Removes legacy depth AE convergence qualification test.
unit-tests/live/d400/test-depth-ae-toggle.py Removes legacy depth AE toggle robustness test.
unit-tests/live/d400/test-d405-calibration-stream.py Removes legacy D405 calibration stream test.
unit-tests/live/d400/test-auto-limits.py Removes legacy auto-exposure/gain limit toggle tests.
unit-tests/live/d400/pytest-pipeline-set-device.py Converts to pytest style using test_device and assert.
unit-tests/live/d400/pytest-mipi-motion.py Adds pytest version of MIPI IMU frame index test with Jetson gating + retries.
unit-tests/live/d400/pytest-hdr-sanity.py Adds pytest HDR streaming sanity test.
unit-tests/live/d400/pytest-hdr-long.py Adds pytest port of the HDR long/nightly suite.
unit-tests/live/d400/pytest-emitter-on-off.py Refactors legacy-style pytest file to use fixtures/markers and pytest-check/logging patterns.
unit-tests/live/d400/pytest-emitter-frequency.py Adds pytest port for emitter frequency behavior tests.
unit-tests/live/d400/pytest-emitter-frequency-negative.py Adds pytest negative support-matrix test for emitter frequency.
unit-tests/live/d400/pytest-disparity-modulation.py Updates to pytest style and uses pytest.approx for float list comparisons.
unit-tests/live/d400/pytest-depth-ae-toggle.py Adds pytest port of the depth AE toggle robustness test (currently skipped).
unit-tests/live/d400/pytest-depth-ae-mode.py Adds pytest port of depth AE mode behavior tests.
unit-tests/live/d400/pytest-depth-ae-metadata.py Converts legacy initialization into pytest fixtures/markers; keeps tests skipped pending FW issue.
unit-tests/live/d400/pytest-depth-ae-convergence.py Adds pytest port of AE convergence qualification test with fixture-based setup/teardown.
unit-tests/live/d400/pytest-d405-calibration-stream.py Adds pytest port of D405 calibration streaming tests using test_context.
unit-tests/live/d400/pytest-auto-limits.py Adds pytest port of auto-exposure/gain limit toggle tests.
unit-tests/conftest.py Adds guarded pyrsutils import.
Comments suppressed due to low confidence (3)

unit-tests/live/d400/pytest-emitter-on-off.py:96

  • This early return only provides 3 values, but the function contract and all call-sites unpack 5 values. This will raise a ValueError when no depth profile is found. Return a 5-tuple with sensible defaults (or pytest.skip/pytest.fail) to keep the API consistent.
    unit-tests/live/d400/pytest-emitter-on-off.py:100
  • This early return only provides 3 values, but the function contract and call-sites unpack 5 values. This will raise a ValueError when no IR profile is found. Return a 5-tuple with sensible defaults (or pytest.skip/pytest.fail) to keep the API consistent.
    unit-tests/live/d400/pytest-pipeline-set-device.py:32
  • pipe.stop() is not in a try/finally. If anything raises after pipe.start(cfg), the pipeline can remain streaming and interfere with later tests. Consider wrapping start/inspection in try/finally and always stopping the pipeline.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread unit-tests/live/d400/pytest-hdr-long.py Resolved
Comment thread unit-tests/live/d400/pytest-hdr-long.py Resolved
@remibettan
remibettan requested a review from Nir-Az April 23, 2026 08:39
f"Spike rate should be < {MAX_ACCEPTABLE_SPIKE_RATE_PERCENT}%; got {spike_rate:.1f}%")

# Compare with baseline
if 'baseline_spike_count' in locals():

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is dropped?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - renamed to pytest-depth-ae-toggle.py as part of the migration. Same test content, new naming convention.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

restored - good catch!

ACCELERATED = 1.0


@pytest.fixture(autouse=True)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

● The default scope is function — it runs once per test function, not per module/file.
To make it run
once per file, you need to set scope explicitly:
"module" │ Once per .py file

@remibettan
remibettan requested a review from Nir-Az April 27, 2026 06:54
remibettan and others added 24 commits April 27, 2026 11:31
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
s1 and s2 were aliased to the same Python object, sharing SW state.
Use ctx.query_devices().front() twice to create independent device
handles, matching the original device_list.front() pattern.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
device() fails when no matching device is found; device_each() generates
0 test instances instead, matching the old infrastructure's skip behavior.
Fixes failure on CI Jetson with D401_GMSL only (no D457 connected).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tests 4/5/6 each re-ran capture_and_analyze_frames twice for calibration,
tripling the streaming work vs legacy. Add a caching contrast_calibration
fixture that measures ON/OFF contrast once per device per module and
returns (contrast_threshold, contrast_diff) to all three tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…n cached function

- pytest-auto-limits.py: add autouse fixture _require_auto_limit_support
  that runs the options/FW version check once per module, removing the
  duplicated per-test guard calls
- pytest-d405-calibration-stream.py: import pytest_check; replace assert
  with check.is_true() inside pipeline frame loops so all iterations
  are validated even on partial failure
- pytest-emitter-on-off.py: convert contrast_calibration pytest fixture
  to plain cached function measure_emitter_contrast(dev, depth_sensor);
  update the three callers
- pytest-mipi-motion.py: import pytest_check; replace assert in the
  frame callback with check.equal() so failures in the callback thread
  are properly reported

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… d400 tests

Adds a cached FW-version gate to rspy/pytest/device_helpers.py:
- require_min_fw_version(dev, min_version, feature_name="", inclusive=True)
- Caches per (serial, version, inclusive) so the check runs at most once per
  device+version combo; subsequent calls are no-ops when already passed.
- Calls pytest.skip() on failure (never writes cache), so every test in the
  module also skips if the device is too old.

Replaces 14 inline fw_version / pytest.skip() blocks across 7 test files:
  pytest-auto-limits, pytest-depth-ae-convergence, pytest-depth-ae-metadata,
  pytest-depth-ae-mode, pytest-depth-ae-toggle, pytest-emitter-frequency,
  pytest-hdr-long.

Also fixes a pre-existing bug in pytest-depth-ae-convergence where the entire
test body was indented inside the fw-check if-block (dead code) — dedented to
function scope so the test actually runs when FW is new enough.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@remibettan
remibettan force-pushed the migrate_d400_folder_to_pytest branch from 5bc70cb to 73003bd Compare April 27, 2026 08:31
remibettan and others added 3 commits April 27, 2026 13:27
… D457

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tion time

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

@Nir-Az Nir-Az left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

@Nir-Az
Nir-Az merged commit feb0d4e into realsenseai:development Apr 28, 2026
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants