Skip to content

[feat] PIP-468: V5 StreamConsumer disconnect / reconnect / past-grace reassignment - #25619

Merged
lhotari merged 3 commits into
apache:masterfrom
merlimat:st-v5-stream-disconnect-tests
Apr 30, 2026
Merged

lhotari merged 3 commits into
apache:masterfrom
merlimat:st-v5-stream-disconnect-tests

Conversation

@merlimat

Copy link
Copy Markdown
Contributor

Summary

Two new V5 StreamConsumer multi-consumer dynamics tests, plus a small broker config to make them fast.

Tests

  • testStreamConsumerReconnectWithinGraceKeepsAssignment — alice + bob share a 4-segment topic. Close alice's consumer and re-subscribe with the same consumerName. The controller re-attaches to the existing session by name (no rebalance, no churn on bob). Same key set across both batches confirms each consumer keeps its segments.
  • testStreamConsumerDisconnectPastGraceTriggersReassignment — same setup, but alice's underlying connection is severed (close her dedicated client) and she stays away past the grace period. Bob ends up serving every key in the second batch. Drain runs concurrently with the rebalance — no Thread.sleep between disconnect and produce.

Wiring

  • ServiceConfiguration.scalableTopicConsumerSessionGracePeriodSeconds (default 60s, matching the previous hard-coded value).
  • ScalableTopicController.createCoordinator reads it and passes it to SubscriptionCoordinator instead of using the static default.
  • SharedPulsarCluster sets it to 2s so reassignment-after-disconnect can be exercised in seconds rather than a minute.

Test plan

  • ./gradlew :pulsar-broker:test --tests \"org.apache.pulsar.client.api.v5.V5MultipleConsumersTest\" — all 5 tests pass; new tests run in ~2.2s and ~3.3s.

Matching PR

  • No matching PR.

area/test

…reassignment

Two new tests for V5 StreamConsumer multi-consumer dynamics:

- testStreamConsumerReconnectWithinGraceKeepsAssignment: alice + bob share
  a 4-segment topic. Force-disconnect alice (close her dedicated client),
  reconnect with the same consumerName well within the controller grace
  period, send a second batch with the same key set. Each consumer's
  per-key share must be identical across batches — proving the controller
  re-attached alice's existing session without rebalancing.
- testStreamConsumerDisconnectPastGraceTriggersReassignment: same setup,
  but alice stays away past the grace period. After the timer fires the
  controller evicts her session and rebalances; bob ends up serving every
  key in the second batch.

Wiring:
- ServiceConfiguration grows scalableTopicConsumerSessionGracePeriodSeconds
  (default 60s, matching the previous hard-coded value).
- ScalableTopicController.createCoordinator passes the config value to the
  SubscriptionCoordinator instead of using the static default.
- SharedPulsarCluster sets it to 2s so reassignment-after-disconnect can
  be exercised in a few seconds rather than a minute.
- Import java.time.Duration in ScalableTopicController instead of
  fully-qualifying inline.
- testStreamConsumerReconnectWithinGraceKeepsAssignment: drop the
  dedicated client + force-disconnect dance. Closing and re-subscribing
  alice's consumer on the shared client is enough to exercise the
  re-attach-existing-session path; the controller looks up the session
  by consumerName.
- testStreamConsumerDisconnectPastGraceTriggersReassignment: drop the
  fixed Thread.sleep before producing. Produce immediately and drain bob
  until every second-batch message lands; the rebalance fires mid-drain
  once the grace timer expires. Cap the drain with a generous deadline
  so a regression fails fast instead of hanging.
@merlimat
merlimat requested a review from lhotari April 29, 2026 17:24
@merlimat merlimat changed the title [test][client] PIP-468: V5 StreamConsumer disconnect / reconnect / past-grace reassignment [client] PIP-468: V5 StreamConsumer disconnect / reconnect / past-grace reassignment Apr 29, 2026
@merlimat merlimat changed the title [client] PIP-468: V5 StreamConsumer disconnect / reconnect / past-grace reassignment [feat] PIP-468: V5 StreamConsumer disconnect / reconnect / past-grace reassignment Apr 29, 2026
…tConfig() is null

ScalableTopicServiceTest.testRegisterConsumerDelegates uses a Mockito-mocked
PulsarService whose getConfig() returns null. The grace-period config lookup
in createCoordinator NPE'd on it. Skip the config read when getConfig() is
null and use the SubscriptionCoordinator's default constructor (60s grace).
@lhotari
lhotari merged commit 1a6b69d into apache:master Apr 30, 2026
43 checks passed
poorbarcode pushed a commit to poorbarcode/pulsar that referenced this pull request May 6, 2026
@lhotari lhotari added this to the 5.0.0-M1 milestone Jun 12, 2026
@merlimat
merlimat deleted the st-v5-stream-disconnect-tests branch October 24, 2026 00:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants