Skip to content

BrotliEncoderSink drops all but the last pulled buffer, silently truncating large Brotli responsesΒ #15432

Closed

Description

@sjanel

Jetty version(s)
12.1.10 (also present in 12.1.11, current latest)

Jetty Environment
core

Java version/vendor
Temurin 21 (Java 17+), Linux x86_64

Description
CompressionHandler with Brotli enabled truncates responses larger than one
encoder input buffer. The client receives a valid, fully-decodable Brotli
stream (no decode error) that contains only a prefix of the payload, so the
truncation is silent β€” decoders succeed and return incomplete data.

gzip and zstd through the same CompressionHandler are unaffected; only Brotli
truncates. Small responses (that fit in a single encoder operation's output)
are fine, which makes this look intermittent in production (only large
responses are cut).

Root cause
In org.eclipse.jetty.compression.brotli.internal.BrotliEncoderSink, the
private drain helper keeps only the last buffer returned by encoder.pull():

protected ByteBuffer encode(EncoderJNI.Operation op)
{
    ...
    ByteBuffer output = null;
    while (true)
    {
        if (!encoder.isSuccess())
            throw new IOException("Brotli Encoder failure");
        else if (encoder.hasMoreOutput())
            output = encoder.pull();          // <-- overwrites; earlier chunks are lost
        else if (encoder.hasRemainingInput())
            encoder.push(op, 0);
        else if (!inputPushed)
        {
            encoder.push(op, inputBuffer.limit());
            inputPushed = true;
        }
        else
        {
            inputBuffer.clear();
            return output;                    // <-- only a single buffer is ever returned
        }
    }
}

EncoderJNI.pull() returns at most one output-buffer-sized chunk. When a single operation (typically the final FLUSH/FINISH after a large body) produces more than one chunk, hasMoreOutput() stays true across multiple pull() calls and every chunk except the last is discarded. The stream still ends with a valid end-of-stream metablock, so it decodes cleanly but short.

The gzip and zstd sinks avoid this: ZstandardEncoderSink returns each output buffer as its own WriteRecord and is re-invoked by EncoderSink, so no output is dropped. BrotliEncoderSink should do the same (emit every pulled buffer) rather than collapsing the drain loop into one returned ByteBuffer.

How to reproduce
Serve a response whose compressed size spans more than one encoder output chunk (a few MB of JSON is more than enough) from a Jetty server with Brotli enabled, then request it with Accept-Encoding: br.

Observed with a ~4.15 MB JSON body:

  • Accept-Encoding: identity β†’ 4,152,362 bytes (complete, valid JSON)
  • Accept-Encoding: gzip β†’ decodes to 4,152,404 bytes (complete, valid JSON)
  • Accept-Encoding: br β†’ decodes without error to 4,128,768 bytes (truncated JSON)

The Brotli response is a clean, complete Brotli stream; only the tail of the payload is missing. The cutoff falls on an encoder input-buffer boundary.

Suggested minimal test (round-trip through the Brotli encoder sink):

byte[] input = /* > a few MB of compressible bytes */;
Compression brotli = new BrotliCompression();
brotli.setByteBufferPool(pool);
brotli.start();

ByteArrayOutputStream collected = new ByteArrayOutputStream();
Content.Sink target = (last, buf, cb) -> { /* append buf to collected */ cb.succeeded(); };
Content.Sink encoder = brotli.newEncoderSink(target, brotli.getDefaultEncoderConfig());
encoder.write(true, ByteBuffer.wrap(input), Callback.NOOP);

byte[] roundTrip = decodeBrotli(collected.toByteArray());
assertArrayEquals(input, roundTrip); // fails: roundTrip is truncated at a buffer boundary

Expected behavior
The decoded Brotli output must be byte-for-byte identical to the original response body, regardless of size.

Actual behavior
For bodies larger than one encoder output chunk, the decoded Brotli output is truncated at an input-buffer boundary (valid stream, incomplete payload).

Activity

  1. added
    BugFor general bugs on Jetty side
    on Jul 16, 2026
  2. added a commit that references this issue on Jul 16, 2026
    030bc56
  3. added a commit that references this issue on Jul 16, 2026
    ea43480
  4. moved this to πŸ— In progress in Jetty 12.1.12 FROZENon Jul 28, 2026
  5. moved this to πŸ— In progress in FROZEN Jetty 12.1.13on Jul 28, 2026
  6. moved this to πŸ— In progress in Jetty 12.1.14 FROZENon Sep 1, 2026
  7. self-assigned this
    on Sep 10, 2026
  8. added a commit that references this issue on Sep 16, 2026
    6ca08ca
  9. moved this from πŸ— In progress to βœ… Done in Jetty 12.1.14 FROZENon Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

BugFor general bugs on Jetty side

Type

No type

Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions