Skip to content

Calling ByteBuf.copy() concurrently from multiple threads will fail #14068

Closed

Description

@lhotari

Expected behavior

Calling .copy() shouldn't mutate the internal state of the ByteBuf instance.
An alternative would be to document this clearly in the Netty javadocs.
The current Javadoc gives the impression that it's safe to call it since it mentions "This method does not modify readerIndex or writerIndex of this buffer."

Actual behavior

Calling .copy() mutates the internal state held in "internalNioBuffer", at least in the case of ReadOnlyByteBufferBuf.
When multiple threads call .copy() concurrently, this leads to various issues such as these stack traces:

java.lang.IllegalArgumentException: newPosition > limit: (2094 > 88)
    at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
    at java.base/java.nio.Buffer.position(Buffer.java:316)
    at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
    at java.base/java.nio.HeapByteBuffer.get(HeapByteBuffer.java:185)
    at io.netty.buffer.UnpooledHeapByteBuf.setBytes(UnpooledHeapByteBuf.java:268)
    at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1113)
    at io.netty.buffer.ReadOnlyByteBufferBuf.copy(ReadOnlyByteBufferBuf.java:431)
    at io.netty.buffer.DuplicatedByteBuf.copy(DuplicatedByteBuf.java:210)
    at io.netty.buffer.AbstractByteBuf.copy(AbstractByteBuf.java:1194)
    at org.apache.pulsar.common.protocol.ByteBufPair$CopyingEncoder.write(ByteBufPair.java:149)
    at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:893)
    at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:875)
    at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:984)
    at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:868)
    at org.apache.pulsar.broker.service.PulsarCommandSenderImpl.lambda$sendMessagesToConsumer$1(PulsarCommandSenderImpl.java:277)
    at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
    at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:413)
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:840)
java.nio.BufferUnderflowException
    at java.base/java.nio.HeapByteBuffer.get(HeapByteBuffer.java:183)
    at io.netty.buffer.UnpooledHeapByteBuf.setBytes(UnpooledHeapByteBuf.java:268)
    at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1113)
    at io.netty.buffer.ReadOnlyByteBufferBuf.copy(ReadOnlyByteBufferBuf.java:431)
    at io.netty.buffer.DuplicatedByteBuf.copy(DuplicatedByteBuf.java:210)
    at io.netty.buffer.AbstractByteBuf.copy(AbstractByteBuf.java:1194)
    at org.apache.pulsar.common.protocol.ByteBufPair$CopyingEncoder.write(ByteBufPair.java:149)
    at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:893)
    at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:875)
    at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:984)
    at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:868)
    at org.apache.pulsar.broker.service.PulsarCommandSenderImpl.lambda$sendMessagesToConsumer$1(PulsarCommandSenderImpl.java:277)
    at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
    at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:413)
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:840)

Steps to reproduce

  • Based on the stack traces, it should be possible to reproduce this by calling .copy() concurrently for a ReadOnlyByteBufferBuf instance.

Minimal yet complete reproducer code (or URL to code)

Netty version

4.1.108

JVM version (e.g. java -version)

openjdk version "17.0.11" 2024-04-16
OpenJDK Runtime Environment Temurin-17.0.11+9 (build 17.0.11+9)
OpenJDK 64-Bit Server VM Temurin-17.0.11+9 (build 17.0.11+9, mixed mode, sharing)

OS version (e.g. uname -a)

Linux pop-os 6.8.0-76060800daily20240311-generic #202403110203171407766522.04~4c8e9a0 SMP PREEMPT_DYNAMIC Thu A x86_64 x86_64 x86_64 GNU/Linux

Activity

  1. lhotari commented on May 22, 2024

    @lhotari
    ContributorAuthor

    #14070 is slightly related.

  2. normanmaurer commented on May 24, 2024

    @normanmaurer
    Member

    Fixed by #14072

  3. added this to the 4.1.111.Final milestone on May 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions