Skip to content

fix(proto): cap bitFieldXXX_ index to prevent OOM in LegacyProtoTypeAdapterFactory - #3027

Merged
eamonnmcmanus merged 3 commits into
google:mainfrom
adilburaksen:fix/legacy-proto-biginteger-oom
May 28, 2026
Merged

eamonnmcmanus merged 3 commits into
google:mainfrom
adilburaksen:fix/legacy-proto-biginteger-oom

Conversation

@adilburaksen

Copy link
Copy Markdown
Contributor

Summary

LegacyProtoTypeAdapterFactory.buildPresenceMaskFromBitFields() reads bitField<N>_ entries from the JSON stream to reconstruct a presence bitmask. The index N was previously used without bound checks: calling BigInteger.shiftLeft(N * 32) with a large attacker-controlled N allocates N / 8 bytes on the heap, making it trivial to OOM the JVM with a small JSON payload (e.g. {"bitField67108863_": 1}).

Fix

Compute maxGroupIndex = ceil(fieldWithPresenceToBitIndex.size() / 32), which is the largest bit-field group index that can influence any presence bit actually checked by this adapter. Any bitField<N>_ entry with N > maxGroupIndex is silently skipped — it cannot affect deserialization correctness and only exists to exhaust memory.

Test

Added deserialize_oversizedBitFieldIndex_doesNotCauseOom to LegacyProtoTypeAdapterFactoryTest, which feeds a crafted JSON with bitField67108863_ and asserts that the valid field optionalInt32_ is still correctly deserialized without any heap pressure.

Impact

Any application that deserializes untrusted JSON using LegacyProtoTypeAdapterFactory is vulnerable. A single JSON field with a large index is enough to trigger the OOM.

…dapterFactory

LegacyProtoTypeAdapterFactory.buildPresenceMaskFromBitFields() reads
bitField<N>_ fields from the JSON stream. If an attacker-supplied JSON
contains an extremely large index (e.g. bitField67108863_), calling
BigInteger.shiftLeft(N * 32) allocates N/8 bytes of heap, making it
trivial to OOM the JVM from a small input.

Fix: compute maxGroupIndex = ceil(fieldWithPresenceToBitIndex.size() / 32)
and skip any bitField group whose index exceeds that bound, since groups
beyond that range cannot affect any presence bit checked by the adapter.

Adds a regression test that verifies a crafted oversized index is silently
ignored while valid fields in the same message are still deserialized.
The BIT_FIELD_PATTERN regex already restricts the captured group to
\d{1,9} (1-9 decimal digits), which guarantees Integer.parseInt succeeds.
Adding an explicit try-catch satisfies static-analysis tools (CodeQL)
that cannot infer the regex-based invariant.
@adilburaksen

Copy link
Copy Markdown
Contributor Author

The CodeQL alert is a valid observation — added a defensive try-catch in the latest commit (a04ecd8). In practice, the BIT_FIELD_PATTERN regex constrains the captured group to \d{1,9}, so parseInt can never throw, but making the safety property explicit is better than relying on the regex invariant.

@Marcono1234

Copy link
Copy Markdown
Contributor

Thanks for highlighting this issue and proposing this fix!

I am wondering if instead of using skipValue() for these cases it would be better to throw JsonParseException. That would also help troubleshooting in case for whatever reason such a JSON property occurs without any malicious intentions.

However, please keep the code as it is for now and let's wait what the maintainers think about this.

@adilburaksen

adilburaksen commented May 27, 2026 •

Copy link
Copy Markdown
Contributor Author

@Marcono1234 Thanks for the review!

Both approaches work to prevent the OOM. I went with skipValue() because it preserves backward compatibility — existing producers that emit large bitField indices (e.g. a newer proto schema with many presence fields) would continue to be parsed rather than throwing. The index beyond maxGroupIndex carries no semantic meaning for deserialization anyway, so silently ignoring it felt safe.

That said, JsonParseException gives a clearer signal for debugging and is arguably the stricter choice. Happy to update if the maintainers prefer that direction.

@eamonnmcmanus eamonnmcmanus left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this is an interesting find! I think, though, that a simpler fix would be to change {1,9} to {1,4} in the regular expression. It seems very unlikely that anyone has a protobuf message with 10,000 fields, and the allocation of a BigInteger to hold 9,999 bits is unlikely to provoke an OOM.

I'm always a bit skeptical about issues arising from deserializing adversarial input, especially for niche cases like this, but I'll note that limiting the size of the BigInteger also mitigates a potential performance DoS due to the quadratic BigInteger.or in the loop.

Replace the dynamic maxGroupIndex guard with the simpler regex approach
suggested in review: limit the captured index to 4 digits (max 9999).
Any real proto message has far fewer than 9999 bitField groups, so the
narrower pattern is a clean fix without added runtime logic.

Also update the test comment to reflect the new mechanism.
@adilburaksen

adilburaksen commented May 28, 2026 •

Copy link
Copy Markdown
Contributor Author

Good call — the regex cap is much cleaner. I went with the dynamic bound initially because I wasn't sure how large the group index could realistically get, but {1,4} is plenty generous for any real proto schema.

Updated: replaced the maxGroupIndex guard with just {1,4} in BIT_FIELD_PATTERN, and reverted the surrounding code back to the original three lines. Also updated the test comment to reflect that the protection now happens at the pattern-matching level rather than at runtime.

@eamonnmcmanus eamonnmcmanus left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for following through on this!

@eamonnmcmanus
eamonnmcmanus merged commit df53d2f into google:main May 28, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants