Skip to content

Reach target parallelism in .limitedParallelism of failing dispatchers - #4330

Merged
dkhalanskyjb merged 2 commits into
developfrom
dk-fix-leaking-limitedParallelism-jobs
Jan 15, 2025
Merged

dkhalanskyjb merged 2 commits into
developfrom
dk-fix-leaking-limitedParallelism-jobs

Conversation

@dkhalanskyjb

Copy link
Copy Markdown
Collaborator

A tiny self-contained piece of #4277

@dkhalanskyjb
dkhalanskyjb requested a review from qwwdfsad January 14, 2025 13:56
dispatch does succeed.
If we don't decrement the counter, it will be impossible to ever reach the target parallelism again. */
runningWorkers.decrementAndGet()
throw e

@JakeWharton JakeWharton Jan 14, 2025 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to basically be a finally block. Any reason not to use that, instead of catch+rethrow? Same holds true below.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason to use catch/throw is that we don't want to deallocate the worker if startWorker finishes normally, as this means the worker is actually started. The same holds true below: if return is used, this means we've already deallocated the worker and don't need to do that again.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, right right. Thanks for clarifying.

@dkhalanskyjb
dkhalanskyjb merged commit ec0d2c6 into develop Jan 15, 2025
@dkhalanskyjb
dkhalanskyjb deleted the dk-fix-leaking-limitedParallelism-jobs branch January 15, 2025 08:53
svc-squareup-copybara pushed a commit to cashapp/misk that referenced this pull request Apr 8, 2025
| Package | Type | Package file | Manager | Update | Change |
|---|---|---|---|---|---|
|
[org.jetbrains.kotlinx:kotlinx-coroutines-test](https://github.com/Kotlin/kotlinx.coroutines)
| dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`1.10.1` -> `1.10.2` |
|
[org.jetbrains.kotlinx:kotlinx-coroutines-slf4j](https://github.com/Kotlin/kotlinx.coroutines)
| dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`1.10.1` -> `1.10.2` |
|
[org.jetbrains.kotlinx:kotlinx-coroutines-core](https://github.com/Kotlin/kotlinx.coroutines)
| dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`1.10.1` -> `1.10.2` |
| [software.amazon.awssdk:sdk-core](https://aws.amazon.com/sdkforjava) |
dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`2.31.16` -> `2.31.17` |
| [software.amazon.awssdk:sqs](https://aws.amazon.com/sdkforjava) |
dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`2.31.16` -> `2.31.17` |
|
[software.amazon.awssdk:dynamodb-enhanced](https://aws.amazon.com/sdkforjava)
| dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`2.31.16` -> `2.31.17` |
| [software.amazon.awssdk:dynamodb](https://aws.amazon.com/sdkforjava) |
dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`2.31.16` -> `2.31.17` |
| [software.amazon.awssdk:aws-core](https://aws.amazon.com/sdkforjava) |
dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`2.31.16` -> `2.31.17` |
| [software.amazon.awssdk:bom](https://aws.amazon.com/sdkforjava) |
dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`2.31.16` -> `2.31.17` |
| [software.amazon.awssdk:auth](https://aws.amazon.com/sdkforjava) |
dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`2.31.16` -> `2.31.17` |

---

### Release Notes

<details>
<summary>Kotlin/kotlinx.coroutines
(org.jetbrains.kotlinx:kotlinx-coroutines-test)</summary>

###
[`v1.10.2`](https://github.com/Kotlin/kotlinx.coroutines/blob/HEAD/CHANGES.md#Version-1102)

[Compare
Source](Kotlin/kotlinx.coroutines@1.10.1...1.10.2)

- Fixed the `kotlinx-coroutines-debug` JAR file including the
`module-info.class` file twice, resulting in failures in various tooling
([#&#8203;4314](Kotlin/kotlinx.coroutines#4314)).
Thanks, [@&#8203;RyuNen344](https://github.com/RyuNen344)!
- Fixed `Flow.stateIn` hanging when the scope is cancelled in advance or
the flow is empty
([#&#8203;4322](Kotlin/kotlinx.coroutines#4322)).
Thanks, [@&#8203;francescotescari](https://github.com/francescotescari)!
- Improved handling of dispatcher failures in `.limitedParallelism`
([#&#8203;4330](Kotlin/kotlinx.coroutines#4330))
and during flow collection
([#&#8203;4272](Kotlin/kotlinx.coroutines#4272)).
- Fixed `runBlocking` failing to run its coroutine to completion in some
cases if its JVM thread got interrupted
([#&#8203;4399](Kotlin/kotlinx.coroutines#4399)).
-   Small tweaks, fixes, and documentation improvements.

</details>

---

### Configuration

πŸ“… **Schedule**: Branch creation - "after 6pm every weekday,before 2am
every weekday" in timezone Australia/Melbourne, Automerge - At any time
(no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

β™» **Rebasing**: Never, or you tick the rebase/retry checkbox.

πŸ‘» **Immortal**: This PR will be recreated if closed unmerged. Get
[config help](https://github.com/renovatebot/renovate/discussions) if
that's undesired.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Renovate
Bot](https://github.com/renovatebot/renovate).

GitOrigin-RevId: 3449fc325ac14573dff5b670e407e99eab23757b
dmitrii-artuhov added a commit to dmitrii-artuhov/kotlinx.coroutines that referenced this pull request May 20, 2025
eizus pushed a commit to cdrxyz/misk that referenced this pull request Jul 18, 2026
| Package | Type | Package file | Manager | Update | Change |
|---|---|---|---|---|---|
|
[org.jetbrains.kotlinx:kotlinx-coroutines-test](https://github.com/Kotlin/kotlinx.coroutines)
| dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`1.10.1` -> `1.10.2` |
|
[org.jetbrains.kotlinx:kotlinx-coroutines-slf4j](https://github.com/Kotlin/kotlinx.coroutines)
| dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`1.10.1` -> `1.10.2` |
|
[org.jetbrains.kotlinx:kotlinx-coroutines-core](https://github.com/Kotlin/kotlinx.coroutines)
| dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`1.10.1` -> `1.10.2` |
| [software.amazon.awssdk:sdk-core](https://aws.amazon.com/sdkforjava) |
dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`2.31.16` -> `2.31.17` |
| [software.amazon.awssdk:sqs](https://aws.amazon.com/sdkforjava) |
dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`2.31.16` -> `2.31.17` |
|
[software.amazon.awssdk:dynamodb-enhanced](https://aws.amazon.com/sdkforjava)
| dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`2.31.16` -> `2.31.17` |
| [software.amazon.awssdk:dynamodb](https://aws.amazon.com/sdkforjava) |
dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`2.31.16` -> `2.31.17` |
| [software.amazon.awssdk:aws-core](https://aws.amazon.com/sdkforjava) |
dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`2.31.16` -> `2.31.17` |
| [software.amazon.awssdk:bom](https://aws.amazon.com/sdkforjava) |
dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`2.31.16` -> `2.31.17` |
| [software.amazon.awssdk:auth](https://aws.amazon.com/sdkforjava) |
dependencies | misk/gradle/libs.versions.toml | gradle | patch |
`2.31.16` -> `2.31.17` |

---

### Release Notes

<details>
<summary>Kotlin/kotlinx.coroutines
(org.jetbrains.kotlinx:kotlinx-coroutines-test)</summary>

###
[`v1.10.2`](https://github.com/Kotlin/kotlinx.coroutines/blob/HEAD/CHANGES.md#Version-1102)

[Compare
Source](Kotlin/kotlinx.coroutines@1.10.1...1.10.2)

- Fixed the `kotlinx-coroutines-debug` JAR file including the
`module-info.class` file twice, resulting in failures in various tooling
([#&#8203;4314](Kotlin/kotlinx.coroutines#4314)).
Thanks, [@&#8203;RyuNen344](https://github.com/RyuNen344)!
- Fixed `Flow.stateIn` hanging when the scope is cancelled in advance or
the flow is empty
([#&#8203;4322](Kotlin/kotlinx.coroutines#4322)).
Thanks, [@&#8203;francescotescari](https://github.com/francescotescari)!
- Improved handling of dispatcher failures in `.limitedParallelism`
([#&#8203;4330](Kotlin/kotlinx.coroutines#4330))
and during flow collection
([#&#8203;4272](Kotlin/kotlinx.coroutines#4272)).
- Fixed `runBlocking` failing to run its coroutine to completion in some
cases if its JVM thread got interrupted
([#&#8203;4399](Kotlin/kotlinx.coroutines#4399)).
-   Small tweaks, fixes, and documentation improvements.

</details>

---

### Configuration

πŸ“… **Schedule**: Branch creation - "after 6pm every weekday,before 2am
every weekday" in timezone Australia/Melbourne, Automerge - At any time
(no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

β™» **Rebasing**: Never, or you tick the rebase/retry checkbox.

πŸ‘» **Immortal**: This PR will be recreated if closed unmerged. Get
[config help](https://github.com/renovatebot/renovate/discussions) if
that's undesired.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Renovate
Bot](https://github.com/renovatebot/renovate).

GitOrigin-RevId: 3449fc325ac14573dff5b670e407e99eab23757b
markharding pushed a commit to futo-org/Harbor that referenced this pull request Oct 13, 2026
This PR contains the following updates:

| Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|
| [org.jetbrains.kotlinx:kotlinx-coroutines-test](https://github.com/Kotlin/kotlinx.coroutines) | `1.10.1` β†’ `1.11.0` | ![age](https://developer.mend.io/api/mc/badges/age/maven/org.jetbrains.kotlinx:kotlinx-coroutines-test/1.11.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/maven/org.jetbrains.kotlinx:kotlinx-coroutines-test/1.10.1/1.11.0?slim=true) |
| [org.jetbrains.kotlinx:kotlinx-coroutines-android](https://github.com/Kotlin/kotlinx.coroutines) | `1.10.1` β†’ `1.11.0` | ![age](https://developer.mend.io/api/mc/badges/age/maven/org.jetbrains.kotlinx:kotlinx-coroutines-android/1.11.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/maven/org.jetbrains.kotlinx:kotlinx-coroutines-android/1.10.1/1.11.0?slim=true) |
| [org.jetbrains.kotlinx:kotlinx-coroutines-core](https://github.com/Kotlin/kotlinx.coroutines) | `1.10.1` β†’ `1.11.0` | ![age](https://developer.mend.io/api/mc/badges/age/maven/org.jetbrains.kotlinx:kotlinx-coroutines-core/1.11.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/maven/org.jetbrains.kotlinx:kotlinx-coroutines-core/1.10.1/1.11.0?slim=true) |

---

### Release Notes

<details>
<summary>Kotlin/kotlinx.coroutines (org.jetbrains.kotlinx:kotlinx-coroutines-test)</summary>

### [`v1.11.0`](https://github.com/Kotlin/kotlinx.coroutines/blob/HEAD/CHANGES.md#Version-1110)

[Compare Source](Kotlin/kotlinx.coroutines@1.10.2...1.11.0)

##### Various

- Kotlin was updated to 2.2.20 ([#&#8203;4545](Kotlin/kotlinx.coroutines#4545)).
- Improved the published jar files ([#&#8203;3842](Kotlin/kotlinx.coroutines#3842), [#&#8203;4599](Kotlin/kotlinx.coroutines#4599)).
- Various documentation improvements, including complete rewrites of structured concurrency and error handling-related KDoc ([#&#8203;4433](Kotlin/kotlinx.coroutines#4433), [#&#8203;4596](Kotlin/kotlinx.coroutines#4596)).

##### Breaking changes and deprecations

- Moved `Promise`-related functions from JS and Wasm/JS to the new `web` target. On Wasm/JS, this is a breaking change. Before the change, `Promise` on Wasm/JS could work with arbitrary Kotlin types, but now, only `JsAny` subtypes are accepted ([#&#8203;4563](Kotlin/kotlinx.coroutines#4563)).
- Changed handling of coroutine exceptions that can't be propagated on JS and Wasm/JS. Before, exceptions were logged, but now, they are reported to the JS runtime ([#&#8203;4451](Kotlin/kotlinx.coroutines#4451), [#&#8203;4631](Kotlin/kotlinx.coroutines#4631)).
- Deprecated using `CoroutineDispatcher` as the coroutine context key; now, `ContinuationInterceptor` has to be used instead ([#&#8203;4333](Kotlin/kotlinx.coroutines#4333)).
- Advanced the deprecation levels on `kotlinx-coroutines-test` APIs ([#&#8203;4604](Kotlin/kotlinx.coroutines#4604)).
- Added lint functions that mark passing a `Job` to coroutine builders as deprecated ([#&#8203;4435](Kotlin/kotlinx.coroutines#4435)).

##### Bug fixes and improvements

- Added a `callsInPlace(EXACTLY_ONCE)` contract to `runBlocking` in code shared between JVM and Native ([#&#8203;4368](Kotlin/kotlinx.coroutines#4368)).
- Added a `callsInPlace(EXACTLY_ONCE)` contract to `suspendCancellableCoroutine` ([#&#8203;4574](Kotlin/kotlinx.coroutines#4574)).
- Fixed `flowOn` incorrectly handling `ThreadContextElement` updates ([#&#8203;4403](Kotlin/kotlinx.coroutines#4403)).
- Fixed exceptions in user-supplied `Thread.UncaughtExceptionHandler` instances causing the internal coroutines machinery to fail ([#&#8203;4516](Kotlin/kotlinx.coroutines#4516)).
- Fixed `CoroutineDispatcher.asScheduler` in the RxJava integration not cancelling outstanding work when a `Worker` gets cancelled, which led to memory leaks in some scenarios ([#&#8203;4615](Kotlin/kotlinx.coroutines#4615)).
- Fixed `SharedFlow` entering an invalid state when a subscriber and an emitter are cancelled simultaneously ([#&#8203;4583](Kotlin/kotlinx.coroutines#4583)).
- Fixed an R8 optimization leading to `shareIn`/`stateIn` coroutines getting garbage-collected ([#&#8203;4646](Kotlin/kotlinx.coroutines#4646)). Thanks, [@&#8203;solevic](https://github.com/solevic)!

##### Small additions

- Added `CompletableDeferred.asDeferred` for obtaining a read-only `Deferred` view ([#&#8203;4408](Kotlin/kotlinx.coroutines#4408)).
- Added `SharedFlow.asFlow` for obtaining a `Flow` view with hidden hot flow semantics ([#&#8203;4530](Kotlin/kotlinx.coroutines#4530)). Thanks, [@&#8203;g000sha256](https://github.com/g000sha256)!
- Added a `StateFlow.collectLatest` overload returning `Nothing` to assist with finding unreachable code ([#&#8203;4454](Kotlin/kotlinx.coroutines#4454)).
- Added `ReceiveChannel.consumeTo` for consuming a `ReceiveChannel` into a `MutableCollection` ([#&#8203;4520](Kotlin/kotlinx.coroutines#4520)).
- Added a `StateFlow<T>.onSubscription` overload returning a `StateFlow<T>`, similar to `SharedFlow<T>.onSubscription` returning `SharedFlow<T>` ([#&#8203;4275](Kotlin/kotlinx.coroutines#4275)). Thanks, [@&#8203;xit0c](https://github.com/xit0c)!
- Added terminal `Flow` operators for collecting a `Flow` to a `Map` ([#&#8203;1541](Kotlin/kotlinx.coroutines#1541)).

##### Changelog relative to version 1.11.0

No changes, only the version is increased.

### [`v1.10.2`](https://github.com/Kotlin/kotlinx.coroutines/blob/HEAD/CHANGES.md#Version-1102)

[Compare Source](Kotlin/kotlinx.coroutines@1.10.1...1.10.2)

- Fixed the `kotlinx-coroutines-debug` JAR file including the `module-info.class` file twice, resulting in failures in various tooling ([#&#8203;4314](Kotlin/kotlinx.coroutines#4314)). Thanks, [@&#8203;RyuNen344](https://github.com/RyuNen344)!
- Fixed `Flow.stateIn` hanging when the scope is cancelled in advance or the flow is empty ([#&#8203;4322](Kotlin/kotlinx.coroutines#4322)). Thanks, [@&#8203;francescotescari](https://github.com/francescotescari)!
- Improved handling of dispatcher failures in `.limitedParallelism` ([#&#8203;4330](Kotlin/kotlinx.coroutines#4330)) and during flow collection ([#&#8203;4272](Kotlin/kotlinx.coroutines#4272)).
- Fixed `runBlocking` failing to run its coroutine to completion in some cases if its JVM thread got interrupted ([#&#8203;4399](Kotlin/kotlinx.coroutines#4399)).
- Small tweaks, fixes, and documentation improvements.

</details>

---

### Configuration

πŸ“… **Schedule**: (UTC)

- Branch creation
  - "before 9am on tuesday"
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

β™» **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

πŸ”• **Ignore**: Close this PR and you won't be reminded about these updates again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xMzMuMCIsInVwZGF0ZWRJblZlciI6IjQ0LjEzMy4wIiwidGFyZ2V0QnJhbmNoIjoiZGV2ZWxvcCIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiLCJyZW5vdmF0ZSJdfQ==-->

Reviewed-on: https://code.futo.org/harbor/harbor/pulls/1829
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants