Repository navigation
NIOFileSystem: saturate ByteCount unit factories instead of trapping on overflow - #3611
Conversation
3bcbffd to
dc28664
Compare
| fileprivate func multipliedSaturating(by other: Int64) -> Int64 { | ||
| let (result, overflow) = self.multipliedReportingOverflow(by: other) | ||
| guard overflow else { | ||
| return result | ||
| } | ||
| // The factors are constant positive multipliers, so the sign of the | ||
| // saturated result follows the sign of `self`. | ||
| return self < 0 ? .min : .max | ||
| } |
There was a problem hiding this comment.
This is fine in this specific case, but just incase someone tries to use this function in another way later, you can remove the 2 line of comments and also ensure the function works in all cases anyway.
Just look at partialValue's (here, result) sign instead of self's:
return result < 0 ? .min : .maxOne note would be that (Int64.min).multipliedReportingOverflow(by: Int64.min) will result in a 0, and in that case, num < 0 ? .min : .max does still behave correctly and counts 0 as a +, which is good.
There was a problem hiding this comment.
I agree with @MahdiBM here, please address this.
There was a problem hiding this comment.
Thanks both. I took the robustness point but landed on a different predicate, because I checked the suggested one and it does not hold.
partialValue is the truncated low 64 bits of the product, so its sign is not the sign of the true product. The clearest counterexample is the one this helper actually hits: Int64.max.multipliedReportingOverflow(by: 1024) reports partialValue == -1024, so result < 0 ? .min : .max would make ByteCount.kibibytes(.max) saturate to Int64.min. I ran both predicates over every overflowing pair drawn from the boundary values and all powers of two, checking against the true product sign computed in arbitrary precision. Out of 8816 overflow cases the partialValue sign is wrong in 4848 of them. Int64.min * Int64.min giving 0 happens to land on the right answer, but that is a coincidence of that one case rather than a general property.
What does hold for any multiplier is the sign of the operands, so I used that and dropped the comment about the factors being positive:
return (self < 0) == (other < 0) ? .max : .minThat is 0 wrong across the same 8816 cases, including negative multipliers, so the helper is now safe to reuse the way you were describing. Pushed, and the target builds clean.
There was a problem hiding this comment.
I can confirm that I stand corrected! @adityasingh2400 thanks for the correction.
| fileprivate func multipliedSaturating(by other: Int64) -> Int64 { | ||
| let (result, overflow) = self.multipliedReportingOverflow(by: other) | ||
| guard overflow else { | ||
| return result | ||
| } | ||
| // The factors are constant positive multipliers, so the sign of the | ||
| // saturated result follows the sign of `self`. | ||
| return self < 0 ? .min : .max | ||
| } |
There was a problem hiding this comment.
I agree with @MahdiBM here, please address this.
β¦on overflow The ByteCount unit factories (kilobytes, megabytes, gigabytes, kibibytes, mebibytes, gibibytes) computed the byte total with a plain multiplication. For large counts such as .gibibytes(.max) that multiplication overflowed Int64 and trapped, crashing the process even though callers reasonably use a very large value to mean no limit. This adds a saturating multiplication helper on Int64 and routes every unit factory through it. Inputs that fit continue to produce the exact same value, and inputs that would overflow now saturate to Int64.max or Int64.min rather than trapping. Tests covering positive and negative overflow are added and both copies of ByteCount.swift are kept in sync. Fixes apple#2877
dc28664 to
1157640
Compare
|
Thanks @MahdiBM, and no worries at all. The @glbrntt this PR is still marked as changes requested from your review on 15 June, which pointed at MahdiBM's suggestion. That suggestion has now been withdrawn above, and the helper on the branch uses the operand-sign predicate instead: return (self < 0) == (other < 0) ? .max : .minThat holds for any multiplier rather than only the positive constants, so the helper is safe to reuse the way you both described. Nothing else is outstanding as far as I can tell, so could you take another look when you have a moment? |
Saturate the
ByteCountunit factories so large counts no longer trap on overflow.Motivation:
ByteCountexposes unit factories such askilobytes,megabytes,gigabytes,kibibytes,mebibytes, andgibibytes. Each one computed the byte total with a plain multiplication, for example1024 * 1024 * 1024 * countingibibytes. When a caller passes a very large count, that multiplication overflowsInt64and traps, crashing the process. This is exactly what issue #2877 reports:ByteCount(contentsOf:maximumSizeAllowed: .gibibytes(.max))crashes with an arithmetic overflow insidegibibytes, even though passing a huge value is a natural way to express "effectively no limit".Modifications:
I added a small
fileprivatesaturating multiplication helper onInt64and routed every unit factory through it. The helper usesmultipliedReportingOverflow(by:)and, on overflow, clamps toInt64.maxfor positive counts orInt64.minfor negative counts. Counts that fit continue to produce the exact same value as before, so there is no behavior change for normal inputs. I extendedByteCountTestswith positive and negative overflow cases, and applied the identical change to both copies ofByteCount.swift(NIOFSand_NIOFileSystem) to keep them in sync, matching the convention used by recent changes to those targets.Result:
Passing extreme values such as
.gibibytes(.max)saturates toInt64.maxinstead of trapping, so disabling a size limit via a large value no longer crashes. I verified this by runningswift test --filter ByteCountTests, which passes after the change. To confirm the tests are load bearing I reverted the source fix and re-ran the new tests, which crashed with signal 5 (the overflow trap) exactly as the issue describes, and they pass again with the fix in place.swift format lint --strictreports no issues on the changed files.Fixes #2877