Skip to content

alts: Release read buffer when blocked on socket read - #8964

Merged
arjan-bal merged 8 commits into
grpc:masterfrom
arjan-bal:alts-read-buffer-pool
Apr 1, 2026
Merged

arjan-bal merged 8 commits into
grpc:masterfrom
arjan-bal:alts-read-buffer-pool

Conversation

@arjan-bal

@arjan-bal arjan-bal commented Mar 9, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Normally, since the Go net.Conn interface provides the abstraction of a blocking read to hide the complexity of non-blocking I/O (epoll, kqueue), users need to pass a read buffer to the net.Conn.Read call. Even when the TCP socket doesn't have data, the application needs to hold onto the read buffer. This results in the ALTS conn and the gRPC HTTP/2 stack holding on to 32KB read buffers each, even for non-readable transports.

Solution

On Unix platforms, there is a RawConn interface that exposes a non-blocking mechanism. The idea is that the Go runtime will call a user-registered callback when the socket is readable. gRPC can use this callback to allocate a buffer from the pool and return it once it has passed the plaintext to the HTTP/2 layer. The same RawConn interface is also available in other OSs, but with slightly different method signatures for the blocking Read method. In the future, we can add the same optimization for them and have CI runners to catch regressions.

The main abstraction that allows these non-memory-pinning reads is the following interface:

// ReadyReader is an optional interface that can be implemented by net.Conn
// implementations to enable gRPC to perform non-memory-pinning reads.
type ReadyReader interface {
	// ReadOnReady waits for data to arrive, fetches a buffer, and performs a
	// read. It returns a pointer to the buffer so you can return it to the pool
	// later.
	ReadOnReady(bufSize int, pool mem.BufferPool) (*[]byte, int, error)
}

In this PR, an implementation is provided that wraps a RawConn. This allows the ALTS conn to perform efficient reads.

In a future PR, the following changes will enable getting rid of the bufio.Reader in the HTTP/2 layer:

  1. ReadyReader will be implemented by the ALTS conn.
  2. gRPC will implement its own buffered reader that releases the buffer when it's empty.
  3. The buffered reader will call ReadOnReady() instead of Read() on the underlying conn, if supported, to delay the re-allocation of the buffer.

Benchmarks

The following micro-benchmarks show no regression in QPS (LargeMessage test) and a significant reduction in memory usage while performing reads (ReadMemoryUsage). There is an increase in 2 allocs in conn construction due to the use of pointer fields for the ReadyReader and read buffer handle, but these happen only when creating a subchannel, not per-RPC. There is an increase in sec/op for the WriteMemoryUsage test, but these tests are not meant to measure conn construction time, only the memory effeciency.

goos: linux
goarch: amd64
pkg: google.golang.org/grpc/credentials/alts/internal/conn
cpu: Intel(R) Xeon(R) CPU @ 2.60GHz
                    β”‚   old.txt   β”‚               new.txt               β”‚
                    β”‚   sec/op    β”‚   sec/op     vs base                β”‚
LargeMessage-48       77.54m Β± 2%   76.51m Β± 1%        ~ (p=0.202 n=15)
WriteMemoryUsage-48   6.816Β΅ Β± 1%   7.519Β΅ Β± 1%  +10.31% (p=0.000 n=15)
ReadMemoryUsage-48    9.754Β΅ Β± 1%   7.021Β΅ Β± 1%  -28.02% (p=0.000 n=15)
geomean               172.7Β΅        159.3Β΅        -7.81%

                    β”‚   old.txt    β”‚               new.txt                β”‚
                    β”‚     B/op     β”‚     B/op      vs base                β”‚
LargeMessage-48       4.578Mi Β± 0%   4.579Mi Β± 7%        ~ (p=0.373 n=15)
WriteMemoryUsage-48   41.60Ki Β± 0%   41.77Ki Β± 0%   +0.39% (p=0.000 n=15)
ReadMemoryUsage-48    83.06Ki Β± 0%   43.25Ki Β± 0%  -47.94% (p=0.000 n=15)
geomean               253.0Ki        203.8Ki       -19.44%

                    β”‚  old.txt   β”‚               new.txt                β”‚
                    β”‚ allocs/op  β”‚ allocs/op   vs base                  β”‚
LargeMessage-48       2.000 Β± 0%   2.000 Β± 0%        ~ (p=1.000 n=15) ΒΉ
WriteMemoryUsage-48   5.000 Β± 0%   7.000 Β± 0%  +40.00% (p=0.000 n=15)
ReadMemoryUsage-48    16.00 Β± 0%   18.00 Β± 0%  +12.50% (p=0.000 n=15)
geomean               5.429        6.316       +16.35%

In a real-world benchmark, where a GCS directpath client downloads a file in a loop, the average "in use" memory falls from ~35MB to ~20MB.

RELEASE NOTES:

  • alts: pool read buffers to lower memory utilization when sockets are unreadable.

@arjan-bal arjan-bal added this to the 1.81 Release milestone Mar 9, 2026
@arjan-bal
arjan-bal requested a review from easwars March 9, 2026 06:39
@arjan-bal arjan-bal added the Type: Performance Performance improvements (CPU, network, memory, etc) label Mar 9, 2026
@arjan-bal arjan-bal changed the title non-blocking read alts: Release read buffer when blocked on socket reads Mar 9, 2026
@codecov

codecov Bot commented Mar 9, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.13043% with 10 lines in your changes missing coverage. Please review.
βœ… Project coverage is 83.09%. Comparing base (b71c262) to head (0cae26e).
⚠️ Report is 5 commits behind head on master.

Files with missing lines Patch % Lines
internal/transport/ready_reader.go 75.00% 5 Missing and 3 partials ⚠️
credentials/alts/internal/conn/record.go 95.65% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #8964      +/-   ##
==========================================
+ Coverage   83.01%   83.09%   +0.08%     
==========================================
  Files         411      413       +2     
  Lines       32960    33051      +91     
==========================================
+ Hits        27361    27464     +103     
+ Misses       4196     4191       -5     
+ Partials     1403     1396       -7     
Files with missing lines Coverage Ξ”
internal/mem/buffer_pool.go 96.52% <100.00%> (+0.09%) ⬆️
internal/transport/raw_conn_linux.go 100.00% <100.00%> (ΓΈ)
credentials/alts/internal/conn/record.go 85.00% <95.65%> (+2.79%) ⬆️
internal/transport/ready_reader.go 75.00% <75.00%> (ΓΈ)

... and 24 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@arjan-bal
arjan-bal force-pushed the alts-read-buffer-pool branch from a937698 to bc023c9 Compare March 9, 2026 09:15
@arjan-bal arjan-bal changed the title alts: Release read buffer when blocked on socket reads alts: Release read buffer when blocked on socket read Mar 9, 2026
@easwars

easwars commented Mar 27, 2026

Copy link
Copy Markdown
Contributor

@arjan-bal : Can you please merge changes from master when you get a chance to get the CI going. Thanks.

@easwars easwars left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't looked much at the alts changes. Should we get someone from the security team to review that?

Comment thread internal/transport/raw_conn_nonlinux.go Resolved
Comment thread internal/transport/ready_reader.go Resolved
Comment thread internal/transport/ready_reader.go Resolved
Comment thread internal/transport/ready_reader.go Resolved
Comment thread internal/transport/ready_reader.go Resolved
Comment thread internal/transport/ready_reader_test.go Resolved
Comment thread internal/transport/ready_reader_test.go Resolved
Comment thread internal/mem/buffer_pool.go Resolved
Comment thread internal/mem/buffer_pool.go Resolved
@easwars easwars assigned arjan-bal and unassigned easwars Mar 27, 2026
@arjan-bal arjan-bal removed their assignment Mar 27, 2026
@arjan-bal
arjan-bal requested a review from gtcooke94 March 27, 2026 10:13
@arjan-bal

Copy link
Copy Markdown
Contributor Author

I haven't looked much at the alts changes. Should we get someone from the security team to review that?

Adding @gtcooke94 for reviewing the ALTS changes.

@arjan-bal
arjan-bal force-pushed the alts-read-buffer-pool branch from b169d72 to f226417 Compare March 27, 2026 10:27

@easwars easwars left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for the non-alts files

@easwars easwars removed their assignment Mar 31, 2026
@arjan-bal

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a ReadyReader interface and a SimpleBufferPool to optimize memory usage during ALTS record reads by avoiding long-term memory pinning when connections are idle. It includes platform-specific implementations for non-blocking reads on Unix systems and updates the ALTS connection logic to utilize these pooled buffers. The review feedback correctly identifies several compilation errors in the new test file ready_reader_test.go where values received from testutils.Channel (which return interface{}) are compared directly to integers without necessary type assertions.

Comment thread internal/transport/ready_reader_test.go Resolved
Comment thread internal/transport/ready_reader_test.go Resolved
Comment thread internal/transport/ready_reader_test.go Resolved
@arjan-bal
arjan-bal force-pushed the alts-read-buffer-pool branch from a14f26a to b6b7d6c Compare March 31, 2026 17:44
@arjan-bal
arjan-bal requested a review from gtcooke94 March 31, 2026 18:01
@arjan-bal arjan-bal assigned arjan-bal and unassigned gtcooke94 Mar 31, 2026
@arjan-bal
arjan-bal force-pushed the alts-read-buffer-pool branch from ef4bb79 to 9636c85 Compare March 31, 2026 18:24
@arjan-bal arjan-bal assigned gtcooke94 and unassigned arjan-bal Mar 31, 2026
@arjan-bal arjan-bal assigned arjan-bal and unassigned gtcooke94 Apr 1, 2026
@arjan-bal
arjan-bal merged commit c33aa45 into grpc:master Apr 1, 2026
13 of 14 checks passed
@arjan-bal
arjan-bal deleted the alts-read-buffer-pool branch April 1, 2026 07:00
arjan-bal added a commit that referenced this pull request Apr 1, 2026
## Problem
The `ReadyReader` interface was added to support non-blocking reads
using the `syscall` package in #8964. While working on [follow-up
changes](https://github.com/grpc/grpc-go/compare/master...arjan-bal:h2-read-buffer-pool?expand=1)
to support pooling read buffers in the http2 layer, I noticed test
failures where servers were closing the TCP connection, but the client
channel remained in the READY state.

## Root Cause
The [unix.Read](https://pkg.go.dev/golang.org/x/sys/unix#Read) function
doesn't consider the graceful closure of a connection to be an error and
instead returns `(0, nil)`. Since consumers of the `io.Reader` interface
expect an `io.EOF` error in such cases, the translation from `(0, nil)`
to `(0, io.EOF)` needs to be handled by the reader implementation.

RELEASE NOTES: N/A
arjan-bal added a commit that referenced this pull request Apr 22, 2026
## Problem
The HTTP/2 framer in gRPC uses a `bufio.Reader` with a 32KB buffer by
default. When there are a large number of transports, these buffers
consume significant memory, even when the transport is idle.

## Solution
#8964 added a `ReadyReader` interface that allows non-memory-pinning
reads. This PR replaces the standard `bufio.Reader` with a custom
`io.Reader` implementation that uses pooled buffers and releases the
buffer once all data is consumed.

To defer the re-allocation of the read buffer, the reader calls
`ReadOnReady` on the underlying `io.Reader`. For this to work, the
underlying `io.Reader` must implement either the `ReadyReader` interface
or `syscall.RawConn`. If neither condition is met, the framer gracefully
falls back to using the regular `bufio.Reader`.

Additional Changes:
* The ALTS connection has been refactored to implement the `ReadyReader`
interface.
* The write buffer pools used by the framer are updated to use the
`mem.BufferPool` interface, allowing the pools to be shared across both
read and write operations.
* Use `syscall.Read` instead of `unix.Read` to avoid triggering the race
detector, see comment for details.
* Add environment variable protection for the changes to allow fast
rollback.

## Benchmarks
In a [real-world
benchmark](https://github.com/arjan-bal/custom-go-client-benchmark/tree/retry-dp),
where a GCS directpath client downloads a file in a loop, the average
"in use" memory falls from 28.3MB to 21.3MB (-24%).

Local Benchmarks show no significant difference
```
❯ go run benchmark/benchresult/main.go streaming-before streaming-after                 
streaming-networkMode_Local-bufConn_false-keepalive_false-benchTime_2m0s-trace_false-latency_0s-kbps_0-MTU_0-maxConcurrentCa
lls_120-reqSize_1024B-respSize_1024B-compressor_off-channelz_false-preloader_false-clientReadBufferSize_-1-clientWriteBuffer
Size_-1-serverReadBufferSize_-1-serverWriteBufferSize_-1-sleepBetweenRPCs_0s-connections_1-recvBufferPool_simple-sharedWrite
Buffer_true
               Title       Before        After Percentage
            TotalOps     29981273     29966908    -0.05%
             SendOps            0            0      NaN%
             RecvOps            0            0      NaN%
            Bytes/op      4971.06      4971.41     0.00%
           Allocs/op        19.79        19.79     0.00%
             ReqT/op 2046721570.13 2045740919.47    -0.05%
            RespT/op 2046721570.13 2045740919.47    -0.05%
            50th-Lat    461.523Β΅s    460.906Β΅s    -0.13%
            90th-Lat    654.435Β΅s    655.327Β΅s     0.14%
            99th-Lat   1.225856ms   1.240984ms     1.23%
             Avg-Lat    478.845Β΅s    479.553Β΅s     0.15%
           GoVersion     go1.25.0     go1.25.0
         GrpcVersion   1.81.0-dev   1.81.0-dev
```

RELEASE NOTES:
* transport: Pool HTTP/2 framer read buffers to reduce idle memory
consumption. Currently limited to Linux for ALTS and non-encrypted
transports (TCP, Unix). To disable, set
`GRPC_GO_EXPERIMENTAL_HTTP_FRAMER_READ_BUFFER_POOLING=false` and report
any issues.
goingforstudying-ctrl added a commit to goingforstudying-ctrl/grpc-go that referenced this pull request Jun 21, 2026
## Problem
Normally, since the Go `net.Conn` interface provides the abstraction of
a blocking read to hide the complexity of non-blocking I/O (epoll,
kqueue), users need to pass a read buffer to the `net.Conn.Read` call.
Even when the TCP socket doesn't have data, the application needs to
hold onto the read buffer. This results in the ALTS conn and the gRPC
HTTP/2 stack holding on to 32KB read buffers each, even for non-readable
transports.

## Solution
On Unix platforms, there is a
[RawConn](https://pkg.go.dev/syscall#RawConn#Read) interface that
exposes a non-blocking mechanism. The idea is that the Go runtime will
call a user-registered callback when the socket is readable. gRPC can
use this callback to allocate a buffer from the pool and return it once
it has passed the plaintext to the HTTP/2 layer. The same RawConn
interface is also available in other OSs, but with slightly different
method signatures for the blocking `Read` method. In the future, we can
add the same optimization for them and have CI runners to catch
regressions.

The main abstraction that allows these non-memory-pinning reads is the
following interface:
```go
// ReadyReader is an optional interface that can be implemented by net.Conn
// implementations to enable gRPC to perform non-memory-pinning reads.
type ReadyReader interface {
	// ReadOnReady waits for data to arrive, fetches a buffer, and performs a
	// read. It returns a pointer to the buffer so you can return it to the pool
	// later.
	ReadOnReady(bufSize int, pool mem.BufferPool) (*[]byte, int, error)
}
```
In this PR, an implementation is provided that wraps a `RawConn`. This
allows the ALTS conn to perform efficient reads.

In a future PR, the following changes will enable getting rid of the
bufio.Reader in the HTTP/2 layer:
1. `ReadyReader` will be implemented by the ALTS conn.
2. gRPC will implement its own buffered reader that releases the buffer
when it's empty.
3. The buffered reader will call `ReadOnReady()` instead of `Read()` on
the underlying conn, if supported, to delay the re-allocation of the
buffer.


## Benchmarks

The following micro-benchmarks show no regression in QPS (LargeMessage
test) and a significant reduction in memory usage while performing reads
(ReadMemoryUsage). There is an increase in 2 allocs in conn construction
due to the use of pointer fields for the `ReadyReader` and read buffer
handle, but these happen only when creating a subchannel, not per-RPC.
There is an increase in sec/op for the `WriteMemoryUsage` test, but
these tests are not meant to measure conn construction time, only the
memory effeciency.

```
goos: linux
goarch: amd64
pkg: google.golang.org/grpc/credentials/alts/internal/conn
cpu: Intel(R) Xeon(R) CPU @ 2.60GHz
                    β”‚   old.txt   β”‚               new.txt               β”‚
                    β”‚   sec/op    β”‚   sec/op     vs base                β”‚
LargeMessage-48       77.54m Β± 2%   76.51m Β± 1%        ~ (p=0.202 n=15)
WriteMemoryUsage-48   6.816Β΅ Β± 1%   7.519Β΅ Β± 1%  +10.31% (p=0.000 n=15)
ReadMemoryUsage-48    9.754Β΅ Β± 1%   7.021Β΅ Β± 1%  -28.02% (p=0.000 n=15)
geomean               172.7Β΅        159.3Β΅        -7.81%

                    β”‚   old.txt    β”‚               new.txt                β”‚
                    β”‚     B/op     β”‚     B/op      vs base                β”‚
LargeMessage-48       4.578Mi Β± 0%   4.579Mi Β± 7%        ~ (p=0.373 n=15)
WriteMemoryUsage-48   41.60Ki Β± 0%   41.77Ki Β± 0%   +0.39% (p=0.000 n=15)
ReadMemoryUsage-48    83.06Ki Β± 0%   43.25Ki Β± 0%  -47.94% (p=0.000 n=15)
geomean               253.0Ki        203.8Ki       -19.44%

                    β”‚  old.txt   β”‚               new.txt                β”‚
                    β”‚ allocs/op  β”‚ allocs/op   vs base                  β”‚
LargeMessage-48       2.000 Β± 0%   2.000 Β± 0%        ~ (p=1.000 n=15) ΒΉ
WriteMemoryUsage-48   5.000 Β± 0%   7.000 Β± 0%  +40.00% (p=0.000 n=15)
ReadMemoryUsage-48    16.00 Β± 0%   18.00 Β± 0%  +12.50% (p=0.000 n=15)
geomean               5.429        6.316       +16.35%
```

In a [real-world
benchmark](https://github.com/arjan-bal/custom-go-client-benchmark/tree/retry-dp),
where a GCS directpath client downloads a file in a loop, the average
"in use" memory falls from ~35MB to ~20MB.

RELEASE NOTES:
* alts: pool read buffers to lower memory utilization when sockets are
unreadable.
goingforstudying-ctrl added a commit to goingforstudying-ctrl/grpc-go that referenced this pull request Jun 21, 2026
## Problem
The `ReadyReader` interface was added to support non-blocking reads
using the `syscall` package in grpc#8964. While working on [follow-up
changes](https://github.com/grpc/grpc-go/compare/master...arjan-bal:h2-read-buffer-pool?expand=1)
to support pooling read buffers in the http2 layer, I noticed test
failures where servers were closing the TCP connection, but the client
channel remained in the READY state.

## Root Cause
The [unix.Read](https://pkg.go.dev/golang.org/x/sys/unix#Read) function
doesn't consider the graceful closure of a connection to be an error and
instead returns `(0, nil)`. Since consumers of the `io.Reader` interface
expect an `io.EOF` error in such cases, the translation from `(0, nil)`
to `(0, io.EOF)` needs to be handled by the reader implementation.

RELEASE NOTES: N/A
goingforstudying-ctrl added a commit to goingforstudying-ctrl/grpc-go that referenced this pull request Jun 21, 2026
## Problem
The HTTP/2 framer in gRPC uses a `bufio.Reader` with a 32KB buffer by
default. When there are a large number of transports, these buffers
consume significant memory, even when the transport is idle.

## Solution
grpc#8964 added a `ReadyReader` interface that allows non-memory-pinning
reads. This PR replaces the standard `bufio.Reader` with a custom
`io.Reader` implementation that uses pooled buffers and releases the
buffer once all data is consumed.

To defer the re-allocation of the read buffer, the reader calls
`ReadOnReady` on the underlying `io.Reader`. For this to work, the
underlying `io.Reader` must implement either the `ReadyReader` interface
or `syscall.RawConn`. If neither condition is met, the framer gracefully
falls back to using the regular `bufio.Reader`.

Additional Changes:
* The ALTS connection has been refactored to implement the `ReadyReader`
interface.
* The write buffer pools used by the framer are updated to use the
`mem.BufferPool` interface, allowing the pools to be shared across both
read and write operations.
* Use `syscall.Read` instead of `unix.Read` to avoid triggering the race
detector, see comment for details.
* Add environment variable protection for the changes to allow fast
rollback.

## Benchmarks
In a [real-world
benchmark](https://github.com/arjan-bal/custom-go-client-benchmark/tree/retry-dp),
where a GCS directpath client downloads a file in a loop, the average
"in use" memory falls from 28.3MB to 21.3MB (-24%).

Local Benchmarks show no significant difference
```
❯ go run benchmark/benchresult/main.go streaming-before streaming-after                 
streaming-networkMode_Local-bufConn_false-keepalive_false-benchTime_2m0s-trace_false-latency_0s-kbps_0-MTU_0-maxConcurrentCa
lls_120-reqSize_1024B-respSize_1024B-compressor_off-channelz_false-preloader_false-clientReadBufferSize_-1-clientWriteBuffer
Size_-1-serverReadBufferSize_-1-serverWriteBufferSize_-1-sleepBetweenRPCs_0s-connections_1-recvBufferPool_simple-sharedWrite
Buffer_true
               Title       Before        After Percentage
            TotalOps     29981273     29966908    -0.05%
             SendOps            0            0      NaN%
             RecvOps            0            0      NaN%
            Bytes/op      4971.06      4971.41     0.00%
           Allocs/op        19.79        19.79     0.00%
             ReqT/op 2046721570.13 2045740919.47    -0.05%
            RespT/op 2046721570.13 2045740919.47    -0.05%
            50th-Lat    461.523Β΅s    460.906Β΅s    -0.13%
            90th-Lat    654.435Β΅s    655.327Β΅s     0.14%
            99th-Lat   1.225856ms   1.240984ms     1.23%
             Avg-Lat    478.845Β΅s    479.553Β΅s     0.15%
           GoVersion     go1.25.0     go1.25.0
         GrpcVersion   1.81.0-dev   1.81.0-dev
```

RELEASE NOTES:
* transport: Pool HTTP/2 framer read buffers to reduce idle memory
consumption. Currently limited to Linux for ALTS and non-encrypted
transports (TCP, Unix). To disable, set
`GRPC_GO_EXPERIMENTAL_HTTP_FRAMER_READ_BUFFER_POOLING=false` and report
any issues.
nschloe pushed a commit to live-clones/forgejo that referenced this pull request Jul 23, 2026
…/forgejo) (#13580)

This PR contains the following updates:

| Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|
| [google.golang.org/grpc](https://github.com/grpc/grpc-go) | `v1.79.3` β†’ `v1.82.1` | ![age](https://developer.mend.io/api/mc/badges/age/go/google.golang.org%2fgrpc/v1.82.1?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/google.golang.org%2fgrpc/v1.79.3/v1.82.1?slim=true) |

---

### gRPC-Go: xDS RBAC and HTTP/2 Vulnerabilities
[GHSA-hrxh-6v49-42gf](GHSA-hrxh-6v49-42gf)

<details>
<summary>More information</summary>

#### Details
Multiple security vulnerabilities have been identified and addressed in grpc-go affecting the xDS RBAC authorization engine (internal/xds/rbac) and the HTTP/2 transport server implementation (internal/transport). These vulnerabilities could result in:

- Authorization Bypass (Fail-Open) when translating xDS RBAC policies containing `Metadata` or `RequestedServerName` fields.
- Denial of Service (High CPU Consumption) due to an HTTP/2 Rapid Reset mitigation bypass during client-initiated stream resets.
- Denial of Service (Server Panic) when parsing crafted xDS RBAC policies containing `NOT` rules around unsupported fields.

##### Impact
_What kind of vulnerability is it? Who is impacted?_

##### xDS RBAC Authorization Bypass via `Metadata` & `RequestedServerName` matchers

- Affected Component: xDS RBAC
- Impact: When building policy matchers for gRPC RBAC from xDS configurations, unsupported `permission` and `principal` rules (specifically `Metadata` and `RequestedServerName`) were silently ignored and treated as no-ops.
  - If an authorization policy relied purely on these matchers for access control, treating those rules as no-ops effectively removed the restrictions.
- If these unsupported rules were nested inside logical `NOT` rules (`Permission_NotRule` / `Principal_NotId`) or multi-condition `OR/AND` rules, silently dropping them changed the boolean logic flow of the authorization engine.

As a result, policy evaluation decisions could fail open, allowing unauthorized clients to access protected gRPC services or resources.

##### HTTP/2 Rapid Reset Mitigation Bypass / Denial of Service via Stream Aborts

- Affected Component: HTTP/2 transport
- Impact: Earlier mitigations in grpc-go for HTTP/2 Rapid Reset only applied threshold checks to items that directly resulted in control frames being written back to the wire, such as `SETTINGS` ACKs or server-initiated `RST_STREAM`s.

When a client initiated a rapid flood of stream creation (`HEADERS`) immediately followed by stream termination `RST_STREAM`, items queued up in the control buffer without counting against the transport response frame threshold. An attacker can repeatedly trigger this flood sequence to bypass reader blocking, resulting in high CPU usage, and Denial of Service (DoS).

##### Denial of Service (Panic) in xDS RBAC Engine via Unsupported Fields inside NOT Rules

- Affected Component: xDS RBAC
- Impact: The xDS RBAC policy translators recursively generate matchers for nested rules. When a `NOT` rule wrapped an unsupported or unhandled field (such as `SourcedMetadata`), the recursive step returned an empty matcher. This could result in a runtime panic when the RBAC engine attempts to authorize an incoming request.

An attacker or misconfigured/malicious xDS management server delivering an LDS/RDS update containing a `NOT` rule around an unhandled field causes the gRPC server process to crash immediately (CWE-248 / Denial of Service).

##### Patches
_Has the problem been patched? What versions should users upgrade to?_

All three issues have been fixed in `master` and will be released in 1.82.1 shortly.

##### Workarounds
_Is there a way for users to fix or remediate the vulnerability without upgrading?_

If upgrading grpc-go immediately is not possible, apply the following workarounds based on your deployment architecture:

* For xDS RBAC Vulnerabilities & Panics: Ensure that upstream xDS management servers do not push RBAC policies containing `Metadata`, `RequestedServerName`, or `NOT` rules wrapping unsupported fields (such as `SourcedMetadata`) to grpc-go servers.
* For HTTP/2 Rapid Reset DOS: Configure upstream reverse proxies or load balancers (such as Envoy) with strict HTTP/2 `max_concurrent_streams` limits and active rate limiting on `RST_STREAM` frequency per connection.

##### Severity

  | Vulnerability | Qualitative Severity | Approximate CVSS v3.1 Score | Primary Impact |
  | :--- | :--- | :--- | :--- |
  | **xDS RBAC Authorization Bypass** | **High** | `8.2` | Unauthorized Access / Fail-Open |
  | **HTTP/2 Rapid Reset DOS Bypass** | **High** | `7.5` | High CPU Consumption / Denial of Service |
  | **xDS RBAC Engine Server Panic** | **Medium** | `5.9` | Process Crash / Denial of Service |

#### Severity
- CVSS Score: 8.8 / 10 (High)
- Vector String: `CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:N/VI:H/VA:H/SC:N/SI:N/SA:N`

#### References
- [https://github.com/grpc/grpc-go/security/advisories/GHSA-hrxh-6v49-42gf](https://github.com/grpc/grpc-go/security/advisories/GHSA-hrxh-6v49-42gf)
- [https://github.com/grpc/grpc-go/pull/9236](https://github.com/grpc/grpc-go/pull/9236)
- [https://github.com/grpc/grpc-go/commit/4ea465d4ab98013f72a142fe0fc89c19770b2935](https://github.com/grpc/grpc-go/commit/4ea465d4ab98013f72a142fe0fc89c19770b2935)
- [https://github.com/grpc/grpc-go](https://github.com/grpc/grpc-go)
- [https://github.com/grpc/grpc-go/releases/tag/v1.82.1](https://github.com/grpc/grpc-go/releases/tag/v1.82.1)

This data is provided by [OSV](https://osv.dev/vulnerability/GHSA-hrxh-6v49-42gf) and the [GitHub Advisory Database](https://github.com/github/advisory-database) ([CC-BY 4.0](https://github.com/github/advisory-database/blob/main/LICENSE.md)).
</details>

---

### Release Notes

<details>
<summary>grpc/grpc-go (google.golang.org/grpc)</summary>

### [`v1.82.1`](https://github.com/grpc/grpc-go/releases/tag/v1.82.1): Release 1.82.1

[Compare Source](grpc/grpc-go@v1.82.0...v1.82.1)

### Security

- server: Stop reading from the connection when flooded by HTTP/2 frames.  The default value for this limit is 100 frames, excluding DATA and HEADERS, and may be changed by setting environment variable `GRPC_GO_EXPERIMENTAL_CONTROL_BUFFER_THROTTLE_LIMIT`.
- xds/rbac: Support `Metadata` and `RequestedServerName` permissions matcher fields.  If present in a DENY rule, previously these would be ignored and fail-open.
- xds/rbac: Fix panic when parsing unsupported fields in `NotRule`/`NotId` permissions.
- xds/rbac: Support the deprecated `source_ip` principal identifier by treating it as equivalent to `direct_remote_ip`.

### [`v1.82.0`](https://github.com/grpc/grpc-go/releases/tag/v1.82.0): Release 1.82.0

[Compare Source](grpc/grpc-go@v1.81.1...v1.82.0)

### Behavior Changes

- server: Remove support for `GRPC_GO_EXPERIMENTAL_DISABLE_STRICT_PATH_CHECKING` environment varibale. Strict incoming RPC path validation (which has been the default since `v1.79.3`) can no longer be disabled. ([#&#8203;9112](grpc/grpc-go#9112))
- transport: Add environment variable to change the default max header list size from `16MB` to `8KB`. This may be enabled by setting `GRPC_GO_EXPERIMENTAL_ENABLE_8KB_DEFAULT_HEADER_LIST_SIZE=true`. This will be enabled by default in a subsequent release. ([#&#8203;9019](grpc/grpc-go#9019))
- balancer: Load Balancing policy registry is now case-sensitive.  Set `GRPC_GO_EXPERIMENTAL_CASE_SENSITIVE_BALANCER_REGISTRIES=false` (and file an issue) to revert to case-insensitive behavior. ([#&#8203;9017](grpc/grpc-go#9017))

### New Features

- experimental/stats: Expose a new API, `NewContextWithLabelCallback`, to register a callback that is invoked when telemetry labels are added. ([#&#8203;8877](grpc/grpc-go#8877))
  - Special Thanks: [@&#8203;seth-epps](https://github.com/seth-epps)
- client: Return a portion of the response body in the error message, when the client receives an unexpected non-gRPC HTTP response, to make debugging easier. ([#&#8203;8929](grpc/grpc-go#8929))
  - Special Thanks: [@&#8203;chengxilo](https://github.com/chengxilo)
- server: Add environment variable `GRPC_GO_SERVER_GOROUTINE_LABELS` that controls setting `runtime/pprof.Labels` on goroutines spawned by the server. Set `GRPC_GO_SERVER_GOROUTINE_LABELS=grpc.method=true` to add the `grpc.method` label on goroutines spawned to handle incoming requests. ([#&#8203;9082](grpc/grpc-go#9082))
  - Special Thanks: [@&#8203;dfinkel](https://github.com/dfinkel)

### Bug Fixes

- xds/server: Fix a memory leak of HTTP filter instances occurring when route configurations are updated in-place during a Route Discovery Service (RDS) update. ([#&#8203;9138](grpc/grpc-go#9138))
- grpc: In the deprecated `gzip` Compressor (used via the deprecated `WithCompressor` dial option), enforce the `MaxRecvMsgSize` limit on the decompressed message buffer, preventing excessive memory allocation from highly compressed payloads. ([#&#8203;9114](grpc/grpc-go#9114))
  - Special Thanks: [@&#8203;evilgensec](https://github.com/evilgensec)
- stats/opentelemetry: Record retry attempts, `grpc.previous-rpc-attempts`, at the call level and not the attempt level. ([#&#8203;8923](grpc/grpc-go#8923))
- encoding: Ensure `Close()` is always called on readers returned from `Compressor.Decompress` if possible. ([#&#8203;9135](grpc/grpc-go#9135))
- channelz: Fix the `LastMessageSentTimestamp` and `LastMessageReceivedTimestamp` fields in `SocketMetrics` to ensure they contain correct timestamp values. ([#&#8203;9109](grpc/grpc-go#9109))

### [`v1.81.1`](https://github.com/grpc/grpc-go/releases/tag/v1.81.1): Release 1.81.1

[Compare Source](grpc/grpc-go@v1.81.0...v1.81.1)

### Security

- xds/rbac: Fix a potential authorization bypass caused by incorrectly falling through URI/DNS SANs to Subject Distinguished Name (DN) when matching the authenticated principal name. With this fix, only the first non-empty identity source will be used, as per [gRFC A41](https://github.com/grpc/proposal/blob/master/A41-xds-rbac.md). ([#&#8203;9111](grpc/grpc-go#9111))
  - Special Thanks: [@&#8203;al4an444](https://github.com/al4an444)

### Bug Fixes

- otel: Segregate client and server RPC information used for metrics and traces, to avoid one overwriting the other. ([#&#8203;9081](grpc/grpc-go#9081))

### [`v1.81.0`](https://github.com/grpc/grpc-go/releases/tag/v1.81.0): Release 1.81.0

[Compare Source](grpc/grpc-go@v1.80.0...v1.81.0)

### Behavior Changes

- balancer/rls: Switch gauge metrics to asynchronous emission (once per collection cycle) to reduce telemetry noise and align with other gRPC language implementations. ([#&#8203;8808](grpc/grpc-go#8808))

### Dependencies

- Minimum supported Go version is now 1.25. ([#&#8203;8969](grpc/grpc-go#8969))

### Bug Fixes

- xds: Use the leaf cluster's security config for the TLS handshake instead of the aggregate cluster's config. ([#&#8203;8956](grpc/grpc-go#8956))
- transport: Send a `RST_STREAM` when receiving an `END_STREAM` when the stream is not already half-closed. ([#&#8203;8832](grpc/grpc-go#8832))
- xds: Fix ADS resource name validation to prevent a panic. ([#&#8203;8970](grpc/grpc-go#8970))

### New Features

- grpc/stats: Add support for custom labels in per-call metrics ([gRFC A108](https://github.com/grpc/proposal/blob/master/A108-otel-custom-per-call-label.md)). ([#&#8203;9008](grpc/grpc-go#9008))
- xds: Add support for Server Name Indication (SNI) and SAN validation ([gRFC A101](https://github.com/grpc/proposal/blob/master/A101-SNI-setting-and-SNI-SAN-validation.md)). Disabled by default. To enable, set `GRPC_EXPERIMENTAL_XDS_SNI=true` environment variable. ([#&#8203;9016](grpc/grpc-go#9016))
- xds: Add support to control which fields get propagated from ORCA backend metric reports to LRS load reports ([gRFC A85](https://github.com/grpc/proposal/blob/master/A85-lrs-custom-metrics-changes.md)). Disabled by default. To enable, set `GRPC_EXPERIMENTAL_XDS_ORCA_LRS_PROPAGATION=true`. ([#&#8203;9005](grpc/grpc-go#9005))
- xds: Add metrics to track xDS client connectivity and cached resource state ([gRFC A78](https://github.com/grpc/proposal/blob/master/A78-grpc-metrics-wrr-pf-xds.md)). ([#&#8203;8807](grpc/grpc-go#8807))
- stats/otel: Enhance `grpc.subchannel.disconnections` metric by adding disconnection reason to the `grpc.disconnect_error` label ([gRFC A94](https://github.com/grpc/proposal/blob/master/A94-subchannel-otel-metrics.md)). This provides granular insights into why subchannels are closing. ([#&#8203;8973](grpc/grpc-go#8973))
- mem: Add `mem.Buffer.Slice()` API to slice the buffer like a slice. ([#&#8203;8977](grpc/grpc-go#8977))
  - Special Thanks: [@&#8203;ash2k](https://github.com/ash2k)

### Performance Improvements

- alts: Pool read buffers to lower memory utilization when sockets are unreadable. ([#&#8203;8964](grpc/grpc-go#8964))
- transport: Pool HTTP/2 framer read buffers to reduce idle memory consumption. Currently limited to Linux for ALTS and non-encrypted transports (TCP, Unix). To disable, set `GRPC_GO_EXPERIMENTAL_HTTP_FRAMER_READ_BUFFER_POOLING=false` and report any issues. ([#&#8203;9032](grpc/grpc-go#9032))

### [`v1.80.0`](https://github.com/grpc/grpc-go/releases/tag/v1.80.0): Release 1.80.0

[Compare Source](grpc/grpc-go@v1.79.3...v1.80.0)

### Behavior Changes

- balancer: log a warning if a balancer is registered with uppercase letters, as balancer names should be lowercase. In a future release, balancer names will be treated as case-insensitive; see [#&#8203;5288](grpc/grpc-go#5288) for details. ([#&#8203;8837](grpc/grpc-go#8837))
- xds: update resource error handling and re-resolution logic ([#&#8203;8907](grpc/grpc-go#8907))
  - Re-resolve all `LOGICAL_DNS` clusters simultaneously when re-resolution is requested.
  - Fail all in-flight RPCs immediately upon receipt of listener or route resource errors, instead of allowing them to complete.

### Bug Fixes

- xds: support the LB policy configured in `LOGICAL_DNS` cluster resources instead of defaulting to `pick_first`. ([#&#8203;8733](grpc/grpc-go#8733))
- credentials/tls: perform per-RPC authority validation against the leaf certificate instead of the entire peer certificate chain. ([#&#8203;8831](grpc/grpc-go#8831))
- xds: enabling A76 ring hash endpoint keys no longer causes EDS resources with invalid proxy metadata to be NACKed when HTTP CONNECT (gRFC A86) is disabled. ([#&#8203;8875](grpc/grpc-go#8875))
- xds: validate that the sum of endpoint weights in a locality does not exceed the maximum `uint32` value. ([#&#8203;8899](grpc/grpc-go#8899))
  - Special Thanks: [@&#8203;RAVEYUS](https://github.com/RAVEYUS)
- xds: fix incorrect proto field access in the weighted round robin (WRR) configuration where `blackout_period` was used instead of `weight_expiration_period`. ([#&#8203;8915](grpc/grpc-go#8915))
  - Special Thanks: [@&#8203;gregbarasch](https://github.com/gregbarasch)
- xds/rbac: handle addresses with ports in IP matchers. ([#&#8203;8990](grpc/grpc-go#8990))

### New Features

- ringhash: enable gRFC A76 (endpoint hash keys and request hash headers) by default. ([#&#8203;8922](grpc/grpc-go#8922))

### Performance Improvements

- credentials/alts: pool write buffers to reduce memory allocations and usage. ([#&#8203;8919](grpc/grpc-go#8919))
- grpc: enable the use of pooled write buffers for buffering HTTP/2 frame writes by default. This reduces memory usage when connections are idle. Use the [WithSharedWriteBuffer](https://pkg.go.dev/google.golang.org/grpc#WithSharedWriteBuffer) dial option or the [SharedWriteBuffer](https://pkg.go.dev/google.golang.org/grpc#SharedWriteBuffer) server option to disable this feature. ([#&#8203;8957](grpc/grpc-go#8957))
- xds/priority: stop caching child LB policies removed from the configuration. This will help reduce memory and cpu usage when localities are constantly switching between priorities. ([#&#8203;8997](grpc/grpc-go#8997))
- mem: add a faster tiered buffer pool; use the experimental [mem.NewBinaryTieredBufferPool](https://pkg.go.dev/google.golang.org/grpc/mem@master#NewBinaryTieredBufferPool) function to create such pools. ([#&#8203;8775](grpc/grpc-go#8775))

</details>

---

### Configuration

πŸ“… **Schedule**: (UTC)

- Branch creation
  - At any time (no schedule defined)
- Automerge
  - Between 12:00 AM and 03:59 AM (`* 0-3 * * *`)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

β™» **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

πŸ”• **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yNzIuMCIsInVwZGF0ZWRJblZlciI6IjQzLjI3Mi4wIiwidGFyZ2V0QnJhbmNoIjoidjE2LjAvZm9yZ2VqbyIsImxhYmVscyI6WyJkZXBlbmRlbmN5LXVwZ3JhZGUiLCJ0ZXN0L25vdC1uZWVkZWQiXX0=-->

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/13580
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Oct 2, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Type: Performance Performance improvements (CPU, network, memory, etc)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants