Skip to content

Make viewer stop-stream non-blocking for immediate UI response - #14695

Merged
Nir-Az merged 4 commits into
realsenseai:developmentfrom
sareluzi:async-stop-stream
Mar 8, 2026
Merged

Nir-Az merged 4 commits into
realsenseai:developmentfrom
sareluzi:async-stop-stream

Conversation

@sareluzi

@sareluzi sareluzi commented Feb 8, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Move heavy sensor operations (s->stop(), frame queue drain, s->close()) in subdevice_model::stop() to a background thread via std::async, keeping all UI state updates synchronous for instant visual feedback
  • Gate subdevice_model::play() on wait_for_stop() so the next start-stream blocks until the background stop completes
  • Ensure clean shutdown by waiting for pending background stop in the destructor and before firmware updates

Test plan

  • Start a stream, click stop — UI button should toggle instantly with no freeze
  • Stop then immediately start — start should block briefly then resume streaming correctly
  • Rapid start/stop toggling (5-10 times) — no crash, correct final state
  • Device removal while streaming — clean removal, no hang
  • Start depth + color, stop one — no interference with the other
  • Application exit while streaming — clean shutdown

🤖 Generated with Claude Code

Copilot AI review requested due to automatic review settings February 8, 2026 09:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Make the viewer “stop streaming” path non-blocking by moving heavy sensor shutdown work to a background task, while keeping UI state updates immediate and synchronous.

Changes:

  • Introduce a background stop task (std::future) and wait_for_stop() to join it.
  • Run heavy sensor operations (stop, frame queue drain, close) via std::async.
  • Block play() (and FW update flow / destructor) on wait_for_stop() to ensure clean transitions.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
common/subdevice-model.h Adds wait_for_stop() API and stores _stop_future to track background stop work
common/subdevice-model.cpp Implements async stop workflow; ensures destructor/play wait for pending stop completion
common/fw-update-helper.cpp Waits for all subdevices’ background stop tasks to complete before firmware update
Comments suppressed due to low confidence (1)

common/subdevice-model.cpp:1

  • std::async runs this lambda on a background thread but it mutates/iterates subdevice_model state via this (_options_invalidated, depth_sensors). If those members are accessed on the UI thread (or other threads) concurrently, this introduces data races/UB. Consider avoiding this capture by copying all required data into the task (e.g., snapshot the relevant sensors/queues before launching), and/or protecting shared members with a mutex/atomics so cross-thread reads/writes are synchronized.
// License: Apache 2.0. See LICENSE file in root directory.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread common/subdevice-model.cpp Outdated
Comment on lines +1500 to +1501
// --- Heavy operations (background) ---
wait_for_stop(); // Wait for any prior pending stop

Copilot AI Feb 8, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although UI flags are flipped before this, stop() can still block the caller/UI thread if a previous background stop is still running (e.g., rapid stop/stop or stop during shutdown). To preserve the PR goal of a non-blocking stop handler, consider making repeated stop() calls return immediately when a stop is already in-flight (e.g., check readiness with wait_for(0)), or serialize stops via a dedicated worker thread/queue rather than blocking here.

Copilot uses AI. Check for mistakes.
Comment thread common/subdevice-model.cpp Outdated
Comment on lines +1519 to +1521
}
catch (...) {}
});

Copilot AI Feb 8, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Catching and swallowing all exceptions here makes failures (e.g., stop()/close() throwing) silently disappear, which can leave the app in a misleading state and complicate debugging. Prefer at least logging the exception (or letting it propagate into the future and handling/reporting it in wait_for_stop()), so failures are observable.

Copilot uses AI. Check for mistakes.
Comment on lines +1468 to +1472
void subdevice_model::wait_for_stop()
{
if (_stop_future.valid())
_stop_future.get();
}

Copilot AI Feb 8, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

std::future is not safe to access concurrently. With wait_for_stop() now callable from multiple code paths (e.g., UI thread play()/stop() and FW update flow), concurrent calls to valid()/get() and assignment to _stop_future can race. Guard _stop_future with a mutex (covering valid/get and the assignment in stop()), or redesign to use a single-threaded stop worker and a thread-safe signaling mechanism.

Copilot uses AI. Check for mistakes.
Move the heavy sensor operations (sensor stop, frame queue drain, sensor
close) in subdevice_model::stop() to a background thread via std::async,
while keeping all UI state updates synchronous. This eliminates the
perceived freeze when the user clicks stop. The next play() call gates
on wait_for_stop() to ensure the sensor is fully closed before reopening.

Details:
- Rapid stop-stop calls chain via future move rather than blocking the
  UI thread, so stop() never blocks the caller.
- _stop_future access is protected by _stop_mutex against concurrent
  callers.
- Exceptions in the background task are logged as warnings via
  viewer.not_model->add_log() instead of being silently swallowed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- wait_for_stop(): move future out under lock, then wait without holding
  the mutex, so the blocking get() does not prevent other callers from
  checking _stop_future.
- Add null check on viewer.not_model before logging in the background
  stop catch handler.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@sareluzi

sareluzi commented Feb 8, 2026

Copy link
Copy Markdown
Collaborator Author

Addressing Aikido review comments (fixed in b5cdede)

1. Null check on viewer.not_model (comment)
Fixed — added if (viewer.not_model) guard before the add_log() call in the background catch handler.

2. Deadlock risk in wait_for_stop() (comment)
Fixed — wait_for_stop() now moves the future out under the lock and calls .get() after releasing it, so the mutex is never held during the blocking wait.

3. Deadlock risk in stop() holding mutex across std::async (comment)
No change needed — std::async returns immediately (it spawns a thread and gives back the future); the heavy work runs asynchronously after stop() returns and the lock is released. The lock must cover both the std::move of the old future and the assignment of the new one to keep _stop_future in a consistent state. No deadlock is possible since the background task never acquires _stop_mutex.

@sareluzi

sareluzi commented Feb 8, 2026

Copy link
Copy Markdown
Collaborator Author

Addressing Copilot review comments (fixed in 39e3a29)

1. stop() blocking UI on rapid stop-stop (comment)
Fixed — stop() no longer calls wait_for_stop() directly. Instead, it moves the previous future into the new async lambda which chains them internally, so stop() never blocks the caller.

2. Silent exception swallowing (comment)
Fixed — exceptions are now logged via viewer.not_model->add_log() with RS2_LOG_SEVERITY_WARN. A bare catch (...) remains as a final fallback for non-std exceptions.

3. _stop_future not thread-safe (comment)
Fixed — added _stop_mutex protecting all _stop_future access (both in wait_for_stop() and stop()).

@Nir-Az

Nir-Az commented Feb 8, 2026

Copy link
Copy Markdown
Collaborator

I need to play with it a bit but I think some corner cases can happen here.
What happens if I click on stop and it take a few seconds and in this time I click on start?
or click on controls I shouldn't click while streaming?

I think that it might have side affects

@sareluzi

sareluzi commented Feb 9, 2026

Copy link
Copy Markdown
Collaborator Author

@Nir-Az - this is why we have wait_for_stop

@sareluzi

sareluzi commented Feb 9, 2026

Copy link
Copy Markdown
Collaborator Author

Re: corner cases (@Nir-Az)

Good questions — here's how each scenario is handled:

1. Click stop, then immediately click start

Handled by design. play() calls wait_for_stop() as its very first operation, so it blocks until the background stop fully completes (s->stop() + queue drain + s->close()) before calling s->open() + s->start(). The user sees the start button become available immediately (responsive UI), but clicking it will briefly block until the sensor is ready to be reopened.

2. Clicking on controls that are normally locked while streaming

Since streaming = false is set immediately, the UI unlocks resolution/FPS/format selectors and sensor options while the sensor is still shutting down in the background. This is safe because:

  • Resolution / FPS / format selectors — these only store UI selections (ui.selected_res_id, etc.) and don't touch the sensor. The new values only take effect when play() calls s->open(profiles), which first waits for the background stop to complete.

  • Sensor options (exposure, gain, etc.) — theoretically someone could call set_option() while the sensor is shutting down. If it hits after s->close(), the existing error handling in option_model catches and surfaces the error. In practice the background stop completes quickly so this timing window is negligible, and the worst case is a transient error message rather than a crash or corruption.

@Nir-Az
Nir-Az requested a review from OhadMeir February 17, 2026 09:03
@Nir-Az

Nir-Az commented Feb 17, 2026

Copy link
Copy Markdown
Collaborator

@OhadMeir please review and try it out.
We need to make sure no locks/racing/null ptr's access and such
Please try some start/stop stress test in the UI as this is a risky change

Comment thread common/fw-update-helper.cpp Resolved
Comment thread common/subdevice-model.cpp Resolved
Comment thread common/subdevice-model.cpp Resolved

@OhadMeir OhadMeir left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Calibration flows call stop then start. Should they call wait_for_stop? I think they are running not on GUI thread. Should check

Comment thread common/subdevice-model.cpp Resolved
@sareluzi

sareluzi commented Mar 3, 2026

Copy link
Copy Markdown
Collaborator Author

Re: calibration flows calling stop then start no additional wait_for_stop() calls are needed. play() already calls wait_for_stop() as its very first operation (line 1631), so every stop() play() path is safe by design. The on-chip-calib flows (stop_viewer() start_viewer() _sub->play()) are all covered through this chain.

Per review feedback: remove try/catch from async lambda so exceptions
propagate via the future and surface through the standard viewer error
handling (popup) at each call site.  The destructor wraps its own
wait_for_stop() in try/catch since destructors must not throw.
@Nir-Az

Nir-Az commented Mar 8, 2026

Copy link
Copy Markdown
Collaborator

@OhadMeir I played with it a bit and it looks OK from UI perspective.
Please review and see if any auto start/stop side affects can occur on calibration flows and such

@Nir-Az
Nir-Az merged commit c5d458c into realsenseai:development Mar 8, 2026
26 of 27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants