Skip to content

castComparableNumber narrows an Integer/Long to Float, so 16777217 compares EQUAL to 16777216.0f #7614

Closed

Description

@lvca

Split out of #7609 / PR #7612, where it was found but left out of scope.

The defect

Type.castComparableNumber() brings two numbers to a common type. When one side is a Float and the other an Integer or a Long, it picks Float and narrows the integral operand into float's 24-bit mantissa:

} else if (left instanceof Long) {
  if (right instanceof Float)
    left = left.floatValue();        // 9007199254740993L -> 9.007199E15

That is the same narrow-instead-of-widen shape #5900 fixed for BinaryComparator and that #7609 has just fixed for the Float/Double and Float/BigDecimal pairs - the integral pairs were simply not part of that change.

Reproduction

Type.castComparableNumber(16777217L, 1.0f)   // -> Float(1.6777216E7), the 17 is gone
Type.castComparableNumber(9007199254740993L, 1.0f)  // -> Float(9.007199E15)
Type.castComparableNumber(2147483647, 1.0f)  // Integer.MAX_VALUE -> Float(2.1474836E9)

And the consequence, which is a wrong answer rather than only a lossy conversion:

Type.castComparableNumber(16777217L, 16777216.0f)  // compareTo == 0

16777217 is not 16777216, but every long above 2^24 collapses onto the nearest float, so a comparison against a FLOAT property reports equality for a whole band of distinct integers. >/>=/</<= are wrong across the same band, and an id-shaped LONG compared against a FLOAT column silently matches the wrong records.

Why it is not in #7609

#7609 is about a floating point literal being read at the wrong precision, and its fix widens a Float toward the wider operand. These branches go the other way - they narrow the wider operand toward the Float - and fixing them changes the answer for Integer/Long vs FLOAT comparisons, which deserves its own change and its own regression test. Worth noting the 49-pair sweep added in PR #7612 checks only that no pair throws, not that no pair loses precision, so it would not have caught this.

Suggested fix

Promote both operands to double rather than narrowing the integral one to float, which is what BinaryComparator.compareNarrowIntegral / compareWideningLong already do for the same pair (Double.compare(value1.doubleValue(), toDouble(value2, type2))). double holds every int exactly and every long up to 2^53, so it is strictly better than float here.

That still leaves long above 2^53 inexact. If the contract is meant to be exact for every integral value, the pair has to meet as BigDecimal instead - worth deciding which of the two this method promises, since #7609 has just made castComparableNumber and BinaryComparator agree everywhere else and this is the last pair where they could still drift apart.

Activity

  1. self-assigned this
    on Sep 15, 2026
  2. added this to the 26.10.1 milestone on Sep 15, 2026
  3. added
    in progressThe team is actively working on this issue
    on Sep 15, 2026
  4. added 2 commits that reference this issue on Sep 15, 2026
  5. lvca commented on Sep 16, 2026

    @lvca
    MemberAuthor

    Fixed: comparing a large Integer/Long against a Float could lose precision and report two different numbers as equal (e.g. 16777217 was treated as equal to 16777216.0). Comparisons now widen to double instead of narrowing to float, matching the precision already used elsewhere in comparisons.

    ❤️ Thanks for helping make ArcadeDB better! ArcadeDB is free and open source, sustained by its community. If it's useful to you or your company, please consider becoming a sponsor to keep development going.

  6. removed
    in progressThe team is actively working on this issue
    on Sep 16, 2026
  7. lvca commented on Sep 16, 2026

    @lvca
    MemberAuthor

    Fixed in #7627 (merged into main).

    The bug: comparing a large INTEGER/LONG value against a FLOAT could lose precision and report two different numbers as equal (e.g. 16777217 was seen as equal to 16777216.0f). Comparisons now widen both sides safely instead of narrowing, so this class of false-equality is gone.


    ❤️ Thanks for helping make ArcadeDB better! ArcadeDB is free and open source, sustained by its community. If it's useful to you or your company, please consider becoming a sponsor to keep development going.

  8. added a commit that references this issue on Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugengineCore engine: document/schema APIsql

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions