Skip to content

fix(javascript): chunked push polling offset and batchSize forwarding - #6598

Merged
eric-zaharia merged 3 commits into
mainfrom
fix/js-polling-bug
Jun 24, 2026
Merged

eric-zaharia merged 3 commits into
mainfrom
fix/js-polling-bug

Conversation

@eric-zaharia

@eric-zaharia eric-zaharia commented Jun 22, 2026 •

Copy link
Copy Markdown
Contributor

🧭 What and Why

🎟 JIRA Ticket: API-503

Fixes two independent defects in the JavaScript transformation/push helpers that together caused waitForTasks to silently under-poll pushed tasks.

  1. Offset over-advance in chunkedPush. The poll cursor was advanced on every object iteration rather than only when a new batch was pushed. Once responses.length reached a multiple of waitBatchSize, the responses.length % waitBatchSize === 0 gate stayed true on subsequent non-push iterations, slicing empty ranges and running offset past the real responses — so most pushed tasks were never polled. Slicing an empty range is a no-op, so there was no crash, hence "silent". Only manifests with multiple wait windows (large inputs).

  2. batchSize dropped by two wrappers. saveObjectsWithTransformation and partialUpdateObjectsWithTransformation destructured their options without batchSize, so it never reached chunkedPush and chunking never happened through those entry points (replaceAllObjectsWithTransformation already forwarded it). A single push left nothing for the cursor to overrun, which masked defect 1 in the existing CTS proof.

The fix mirrors the Python reference (#6486): gated wait + bounded cursor + post-loop drain.

Changes included:

  • chunkedPush (templates/javascript/clients/client/api/ingestionHelpers.mustache): extract the polling loop into a waitForBatch(start, end) closure; replace the % waitBatchSize gate with responses.length - offset >= waitBatchSize, advance the cursor by exactly waitBatchSize, and drain the remaining tail after the loop.
  • Wrappers (templates/javascript/clients/algoliasearch/builds/definition.mustache): forward batchSize to chunkedPush in saveObjectsWithTransformation and partialUpdateObjectsWithTransformation.
  • Unskip the chunkedPushWait CTS proof for javascript (tests/CTS/client/search/saveObjectsWithTransformation.json and the assertion count in scripts/cts/runCts.ts).

🧪 Test

Covered by the existing chunkedPushWait CTS scenario — saveObjectsWithTransformation with 25 objects, waitForTasks: true, batchSize: 10 → 3 pushes. The test server asserts getEventCount === pushCount and that the set of distinct polled eventIDs equals pushCount, i.e. every pushed task is polled exactly once. Before this PR the scenario was skipped for JS; it now runs. PHP remains skipped for this scenario.

@eric-zaharia eric-zaharia self-assigned this Jun 22, 2026
@algolia-api-clients-automation-bot

algolia-api-clients-automation-bot Bot commented Jun 22, 2026 •

Copy link
Copy Markdown
Contributor

💥 Breaking changes detected !!

Either this PR or a previous PR not released yet introduced breaking changes, be careful when merging.
You can find the details in the 'client java@21' CI job.

✔️ Code generated!

Name Link
🪓 Triggered by 6ee9d494686c55616653e429a7b7d71a08c2651d
🍃 Generated commit 0a4ebe56cc4246763e99865f8fc361871e3df734
🌲 Generated branch generated/fix/js-polling-bug
📊 Benchmark results

Benchmarks performed on the method using a mock server, the results might not reflect the real-world performance.

Language Req/s
javascript 2298
go 1883
php 1845
csharp 1489
python 1436
java 1103
ruby 953
swift 849
scala 23

@eric-zaharia
eric-zaharia marked this pull request as ready for review June 23, 2026 07:47
@eric-zaharia
eric-zaharia requested a review from a team as a code owner June 23, 2026 07:47

@Fluf22 Fluf22 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only one comment: can you please gate against weird batchSize values (zero or negative)?

@eric-zaharia
eric-zaharia marked this pull request as draft June 24, 2026 06:41
@eric-zaharia
eric-zaharia marked this pull request as ready for review June 24, 2026 07:10
@eric-zaharia
eric-zaharia requested a review from Fluf22 June 24, 2026 07:18
@eric-zaharia
eric-zaharia merged commit d1f8a51 into main Jun 24, 2026
35 checks passed
@eric-zaharia
eric-zaharia deleted the fix/js-polling-bug branch June 24, 2026 12:25
algolia-api-clients-automation-bot Bot added a commit that referenced this pull request Jun 24, 2026
…#6598) (generated) [skip ci]

Co-authored-by: Eric Zaharia <94015633+eric-zaharia@users.noreply.github.com>
algolia-api-clients-automation-bot Bot added a commit to algolia/algoliasearch-client-javascript that referenced this pull request Jun 24, 2026
… (generated)

algolia/api-clients-automation#6598

Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com>
Co-authored-by: Eric Zaharia <94015633+eric-zaharia@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants