Skip to content

Fix for cloning NPU Image Generation pipelines - #2376

Merged
dkalinowski merged 8 commits into
openvinotoolkit:releases/2025/2from
dkalinowski:npu-imagegen-clone-fix
Jun 27, 2025
Merged

dkalinowski merged 8 commits into
openvinotoolkit:releases/2025/2from
dkalinowski:npu-imagegen-clone-fix

Conversation

@dkalinowski

@dkalinowski dkalinowski commented Jun 24, 2025 •

Copy link
Copy Markdown
Contributor

Pipeline cloning for multi-threaded usage had a few bugs:

  • generation config was not copied to newly cloned instances, causing n/h/w/guidance_scale to not propagate further
  • BS1 inference classes clone/copy had issue with default constructors which caused m_requests to duplicate and grow over time

This change was tested with SD v1.5 with text2image_concurrent sample, and this change will enable OpenVINO Model Server with NPU enablement for Image Generation use cases.

CVS-168932

@dkalinowski
dkalinowski requested review from Wovchena and likholat June 24, 2025 10:24
@github-actions github-actions Bot added category: image generation Image generation pipelines category: Image generation samples GenAI Image generation samples labels Jun 24, 2025
@Wovchena
Wovchena requested a review from Copilot June 24, 2025 10:27

This comment was marked as outdated.

dkalinowski and others added 2 commits June 24, 2025 12:46
Co-authored-by: Vladimir Zlobin <vladimir.zlobin@intel.com>
UNetInferenceStaticBS1(const UNetInferenceStaticBS1&) = delete;
UNetInferenceStaticBS1(UNetInferenceStaticBS1&&) = delete;

UNetInferenceStaticBS1(UNetInferenceStaticBS1& other)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should disable direct copying for user, the clone() method must be the only way to create a copy.
Let's move this constructor to the private section

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

pipelines.emplace_back(models_path, device);

properties = ov::AnyMap{ov::genai::strength(0.8f), // controls how initial image is noised after being converted to latent space. `1` means initial image is fully noised
ov::genai::num_inference_steps(2)};

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

num_inference_steps was 4 for that sample (and 2 for samples/cpp/image_generation/text2image_concurrency.cpp). I'd propose to make it 4 for samples/cpp/image_generation/image2image_concurrency.cpp. Unless @likholat says the opposite

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Wovchena is right, we need to do at least 4 steps for FLUX.1-schnell to get a decent image, and more steps for other models

@Wovchena
Wovchena requested a review from Copilot June 26, 2025 12:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes bugs in pipeline cloning for multi-threaded NPU image generation by ensuring that the generation configuration is copied correctly and by addressing issues in BS1 inference cloning. Key changes include:

  • Adding calls to set_generation_config in multiple pipeline types.
  • Revising the clone methods for UNet and SD3Transformer BS1 inference classes to avoid issues caused by default constructors.
  • Updating sample applications to support NPU usage with appropriate properties.

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/cpp/src/image_generation/stable_diffusion_xl_pipeline.hpp Added generation config copy to cloned pipelines
src/cpp/src/image_generation/stable_diffusion_pipeline.hpp Added generation config copy to cloned pipelines
src/cpp/src/image_generation/stable_diffusion_3_pipeline.hpp Added generation config copy to cloned pipelines
src/cpp/src/image_generation/models/unet_inference_static_bs1.hpp Updated clone() implementation for inference with BS1 to prevent duplicated requests
src/cpp/src/image_generation/models/sd3transformer_2d_inference_static_bs1.hpp Updated clone() implementation for inference with BS1 to prevent duplicated requests
src/cpp/src/image_generation/flux_pipeline.hpp Added generation config copy to cloned pipelines
samples/cpp/image_generation/text2image_concurrency.cpp Modified pipeline instantiation and cloning, and updated properties for NPU usage
samples/cpp/image_generation/image2image_concurrency.cpp Modified pipeline instantiation and cloning, and updated properties for NPU usage


UNetInferenceStaticBS1(const UNetInferenceStaticBS1&) = delete;
UNetInferenceStaticBS1(UNetInferenceStaticBS1&&) = delete;
UNetInferenceStaticBS1(UNetInferenceStaticBS1& other) = delete;

Copilot AI Jun 26, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Both the const copy constructor and a non-const copy constructor are deleted; consider if deleting the non-const version is redundant.

Suggested change
UNetInferenceStaticBS1(UNetInferenceStaticBS1& other) = delete;

Copilot uses AI. Check for mistakes.

InferenceStaticBS1(const InferenceStaticBS1&) = delete;
InferenceStaticBS1(InferenceStaticBS1&&) = delete;
InferenceStaticBS1(InferenceStaticBS1& other) = delete;

Copilot AI Jun 26, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Both the const copy constructor and a non-const copy constructor are deleted; consider if deleting the non-const version is redundant.

Suggested change
InferenceStaticBS1(InferenceStaticBS1& other) = delete;

Copilot uses AI. Check for mistakes.
@likholat
likholat added this pull request to the merge queue Jun 26, 2025
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 26, 2025
@Wovchena
Wovchena added this pull request to the merge queue Jun 27, 2025
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 27, 2025
@dkalinowski
dkalinowski added this pull request to the merge queue Jun 27, 2025
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 27, 2025
@dkalinowski
dkalinowski added this pull request to the merge queue Jun 27, 2025
Merged via the queue into openvinotoolkit:releases/2025/2 with commit 0881726 Jun 27, 2025
Wovchena added a commit that referenced this pull request Jul 2, 2025
This is cherry-pick from release branch.

Pipeline cloning for multi-threaded usage had a few bugs:
- generation config was not copied to newly cloned instances, causing
n/h/w/guidance_scale to not propagate further
- BS1 inference classes clone/copy had issue with default constructors
which caused m_requests to duplicate and grow over time

This change was tested with SD v1.5 with `text2image_concurrent` sample,
and this change will enable OpenVINO Model Server with NPU enablement
for Image Generation use cases.

CVS-168932

---------

Co-authored-by: Vladimir Zlobin <vladimir.zlobin@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: Image generation samples GenAI Image generation samples category: image generation Image generation pipelines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants