Repository navigation
Fix for cloning NPU Image Generation pipelines - #2376
dkalinowski merged 8 commits into
Conversation
Co-authored-by: Vladimir Zlobin <vladimir.zlobin@intel.com>
| UNetInferenceStaticBS1(const UNetInferenceStaticBS1&) = delete; | ||
| UNetInferenceStaticBS1(UNetInferenceStaticBS1&&) = delete; | ||
|
|
||
| UNetInferenceStaticBS1(UNetInferenceStaticBS1& other) |
There was a problem hiding this comment.
We should disable direct copying for user, the clone() method must be the only way to create a copy.
Let's move this constructor to the private section
| pipelines.emplace_back(models_path, device); | ||
|
|
||
| properties = ov::AnyMap{ov::genai::strength(0.8f), // controls how initial image is noised after being converted to latent space. `1` means initial image is fully noised | ||
| ov::genai::num_inference_steps(2)}; |
There was a problem hiding this comment.
num_inference_steps was 4 for that sample (and 2 for samples/cpp/image_generation/text2image_concurrency.cpp). I'd propose to make it 4 for samples/cpp/image_generation/image2image_concurrency.cpp. Unless @likholat says the opposite
There was a problem hiding this comment.
@Wovchena is right, we need to do at least 4 steps for FLUX.1-schnell to get a decent image, and more steps for other models
There was a problem hiding this comment.
Pull Request Overview
This PR fixes bugs in pipeline cloning for multi-threaded NPU image generation by ensuring that the generation configuration is copied correctly and by addressing issues in BS1 inference cloning. Key changes include:
- Adding calls to set_generation_config in multiple pipeline types.
- Revising the clone methods for UNet and SD3Transformer BS1 inference classes to avoid issues caused by default constructors.
- Updating sample applications to support NPU usage with appropriate properties.
Reviewed Changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/cpp/src/image_generation/stable_diffusion_xl_pipeline.hpp | Added generation config copy to cloned pipelines |
| src/cpp/src/image_generation/stable_diffusion_pipeline.hpp | Added generation config copy to cloned pipelines |
| src/cpp/src/image_generation/stable_diffusion_3_pipeline.hpp | Added generation config copy to cloned pipelines |
| src/cpp/src/image_generation/models/unet_inference_static_bs1.hpp | Updated clone() implementation for inference with BS1 to prevent duplicated requests |
| src/cpp/src/image_generation/models/sd3transformer_2d_inference_static_bs1.hpp | Updated clone() implementation for inference with BS1 to prevent duplicated requests |
| src/cpp/src/image_generation/flux_pipeline.hpp | Added generation config copy to cloned pipelines |
| samples/cpp/image_generation/text2image_concurrency.cpp | Modified pipeline instantiation and cloning, and updated properties for NPU usage |
| samples/cpp/image_generation/image2image_concurrency.cpp | Modified pipeline instantiation and cloning, and updated properties for NPU usage |
|
|
||
| UNetInferenceStaticBS1(const UNetInferenceStaticBS1&) = delete; | ||
| UNetInferenceStaticBS1(UNetInferenceStaticBS1&&) = delete; | ||
| UNetInferenceStaticBS1(UNetInferenceStaticBS1& other) = delete; |
There was a problem hiding this comment.
[nitpick] Both the const copy constructor and a non-const copy constructor are deleted; consider if deleting the non-const version is redundant.
| UNetInferenceStaticBS1(UNetInferenceStaticBS1& other) = delete; |
|
|
||
| InferenceStaticBS1(const InferenceStaticBS1&) = delete; | ||
| InferenceStaticBS1(InferenceStaticBS1&&) = delete; | ||
| InferenceStaticBS1(InferenceStaticBS1& other) = delete; |
There was a problem hiding this comment.
[nitpick] Both the const copy constructor and a non-const copy constructor are deleted; consider if deleting the non-const version is redundant.
| InferenceStaticBS1(InferenceStaticBS1& other) = delete; |
This is cherry-pick from release branch. Pipeline cloning for multi-threaded usage had a few bugs: - generation config was not copied to newly cloned instances, causing n/h/w/guidance_scale to not propagate further - BS1 inference classes clone/copy had issue with default constructors which caused m_requests to duplicate and grow over time This change was tested with SD v1.5 with `text2image_concurrent` sample, and this change will enable OpenVINO Model Server with NPU enablement for Image Generation use cases. CVS-168932 --------- Co-authored-by: Vladimir Zlobin <vladimir.zlobin@intel.com>
Pipeline cloning for multi-threaded usage had a few bugs:
This change was tested with SD v1.5 with
text2image_concurrentsample, and this change will enable OpenVINO Model Server with NPU enablement for Image Generation use cases.CVS-168932