Skip to content

fix(lib-transfer-manager): respect file read stream's byte range on upload - #8324

Merged
smilkuri merged 1 commit into
mainfrom
fix/tm-upload-offset
Sep 30, 2026
Merged

smilkuri merged 1 commit into
mainfrom
fix/tm-upload-offset

Conversation

@smilkuri

@smilkuri smilkuri commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Description

The threaded file upload path reads the file directly by offset instead of consuming the body stream, so a body with explicit byte range, like createReadStream(path, { start, end }) was uploaded incorrectly.

Testing

added unit test for windowed stream

✓ src/submodules/transfer-manager/join-streams.spec.ts (10 tests) 578ms
     ✓ should handle backpressure when consumer is slow  554ms
 ✓ src/submodules/transfer-manager/S3TransferManager.spec.ts (120 tests) 438ms

 Test Files  2 passed (2)
      Tests  130 passed (130)
   Start at  16:49:44
   Duration  1.04s (transform 441ms, setup 0ms, import 693ms, tests 1.02s, environment 0ms)

Checklist

  • If the PR is a feature, add integration tests (*.integ.spec.ts) or E2E tests.
    • It's not a feature.
  • My E2E tests are resilient to concurrent i/o.
    • I didn't write any E2E tests.
  • I added access level annotations e.g. @public, @internal tags and enabled doc generation on the package. Remember that access level annotations go below the description, not above.
    • I didn't add any public functions.
  • Streams - how do they work?? My WebStream readers/locks are properly lifecycled. Node.js stream backpressure is handled. Error handling.
    • No streams here.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

return input.size;
} else if (typeof input.start === "number" && typeof input.end === "number") {
return input.end + 1 - input.start;
} else if (input instanceof ReadStream) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re-ordered this check. Previously they always matched the generic start/end check first since end defaults to infinity when only { start: 10 } is given and it returned Infinity, and when they did reach here, end was ignored, so { end: 19 } returned the full file size and over-read.

@smilkuri
smilkuri marked this pull request as ready for review September 30, 2026 17:26
@smilkuri
smilkuri requested a review from a team as a code owner September 30, 2026 17:26
@smilkuri
smilkuri merged commit 449c281 into main Sep 30, 2026
11 checks passed
@smilkuri
smilkuri deleted the fix/tm-upload-offset branch September 30, 2026 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants