Repository navigation
[improve] Wire Set<Option> through backend hooks + Oxia PartitionKey - #25723
Merged
Merged
Conversation
…re Oxia PartitionKey Continue the Set<Option> flip down into the backend layer so backends receive routing/hint options on every operation, and wire OxiaMetadataStore to consume Option.PartitionKey natively. API surface gains canonical Set<Option> overloads for the remaining methods: - MetadataStore.deleteRecursive(path, opts) - MetadataStore.deleteIfExists(path, version, opts) AbstractMetadataStore backend hooks all flipped to Set<Option>: - storeGet, storeDelete, getChildrenFromStore, existsFromStore - storeScanChildren, storeFindByIndex LocalMemory, RocksDB, ZK (via AbstractBatchedMetadataStore + OpGet/OpDelete/ OpGetChildren), and Oxia updated to the new signatures. OxiaMetadataStore extracts PartitionKey via OptionsHelper.partitionKey and attaches the matching Oxia option (GetOption / PutOption / DeleteOption / ListOption / RangeScanOption) to every client call. Tests: new OxiaPartitionKeyTest exercises put/get/delete/getChildren round trips against a 3-shard Oxia cluster.
lhotari
self-requested a review
May 8, 2026 17:21
1 task done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Follow-up to #25710 (the additive Set API). That PR landed the public-API surface and the canonical
Set<Option>form on every read/write method, but the backend-side hooks onAbstractMetadataStore(storeGet,storeDelete,getChildrenFromStore,existsFromStore,storeScanChildren,storeFindByIndex) still took only path/version — opts were dropped at theAbstractMetadataStoreboundary. That meantOption.PartitionKey, the routing hint sharded backends like Oxia need, never reached the client.This PR plumbs
Set<Option>end-to-end and wiresOxiaMetadataStoreto consumeOption.PartitionKeynatively.Modifications
Backend hooks:
AbstractMetadataStore.storeGet,storeDelete,getChildrenFromStore,existsFromStore,storeScanChildren, andstoreFindByIndexall takeSet<Option>. Backends override the new signatures; opts ignored where not used (LocalMemory, RocksDB).ZK batching layer:
OpGet,OpDelete,OpGetChildreneach carry the opts so the batch flusher has them available;AbstractBatchedMetadataStoreplumbs opts into the Op constructors.Oxia:
OxiaMetadataStoreextractsPartitionKeyviaOptionsHelper.partitionKeyand attaches the matching client option (GetOption.PartitionKey/PutOption.PartitionKey/DeleteOption.PartitionKey/ListOption.PartitionKey/RangeScanOption.PartitionKey) to every Oxia call. The twostorePutoverloads collapse into one.API completeness:
MetadataStore.deleteRecursiveanddeleteIfExistsgotSet<Option>overloads to round out the surface.AbstractMetadataStore.deleteRecursivethreads opts through every recursive child + the finaldeleteIfExistsso PartitionKey routing applies to the whole subtree.Wrappers (
DualMetadataStore,FaultInjectionMetadataStore): updated to the new signatures.Verifying this change
New
OxiaPartitionKeyTestruns against a 3-shard Oxia cluster and verifies put/get/delete/getChildren round-trips withOption.PartitionKey.The test deliberately asserts only the round-trip — asserting that a record routed to shard(K) is unreachable from shard(J) is statistically flaky on 3 shards because half the (K,J) pairs collide. A stronger correctness assertion would need access to Oxia internals.
Does this pull request potentially affect one of the following parts:
MetadataStoreAPI — strictly additive: new abstractSet<Option>overloads added fordeleteRecursiveanddeleteIfExists, legacy no-opts forms become defaults that forward. Existing callers and external implementations are unaffected.AbstractMetadataStore— these are protected, used only by the in-tree backends, and all in-tree backends are updated in this PR.Matching PR in forked repository
PR in forked repository: https://github.com/merlimat/pulsar/pull/new/mmerli/metadata-partition-key