Skip to content

perf(utils): speed up text_in_bbox (≈10x) and random_string (4x) - #718

Merged
bosd merged 3 commits into
camelot-dev:masterfrom
bosd:perf/text-in-bbox-and-random-string
May 20, 2026
Merged

bosd merged 3 commits into
camelot-dev:masterfrom
bosd:perf/text-in-bbox-and-random-string

Conversation

@bosd

@bosd bosd commented May 19, 2026 •

Copy link
Copy Markdown
Collaborator

Low-hanging fruit from a profiling pass on camelot/utils.py. Both are pure refactors — output is bit-for-bit identical to the previous implementation.

text_in_bbox: O(n³) → O(n²), ≈9.5× faster

The 'discard >80%-contained boxes' phase called rest.copy() inside the outer loop, making it O(n³) in the size of t_bbox. Tracked discards in a side set[int] of indices instead; same semantics, no per-iteration copies.

Microbench (800 synthetic boxes, 1000-unit page):

old: 165.3 ms
new:  17.3 ms   (9.5× faster, 11 kept in both)

This function is called once per (table_area × parse), and n is the count of text objects on a page — typically hundreds, sometimes thousands on dense reports. Expect a noticeable end-to-end win on those PDFs; smaller pages will not regress.

random_string: ≈4× faster, no behavioural change

random.choice in a Python loop with ret += … rebuilds the alphabet on every iteration and allocates a fresh string each step. Replaced with one random.choices(_AL, k=length) + str.join, alphabet cached at module import.

Microbench (10 000 calls of length 32):

old: 203.1 ms
new:  50.7 ms   (4.0× faster)

bosd added 3 commits May 20, 2026 10:38
Two low-hanging hotspots identified during a profiling pass:

* random_string used per-char string concatenation in a while-loop and
  rebuilt the alphabet on every iteration. Switched to random.choices
  + str.join with the alphabet cached at import time. Microbench:
  4.0x faster (10k calls of length 32: 203ms → 51ms).

* text_in_bbox ran an O(n³) discard pass because rest.copy() was
  called inside the outer loop. The same set membership semantics can
  be expressed with a side 'discarded' set of indices and no copies.
  Microbench on 800 synthetic boxes: 9.5x faster (165ms → 17ms), with
  identical kept-set.

Both changes preserve observable behaviour (rendered table content is
unchanged); only allocation/iteration patterns differ. See the perf
report on the PR for the broader profiling investigation.
Adds two unit tests that exercise the refactored hot paths directly:

- test_random_string_length_and_alphabet covers the new
  random.choices + str.join path, including the length=0 edge case
  that the previous while-loop hit by accident (loop body skipped).
- test_text_in_bbox_filters_and_discards_overlaps covers the new
  O(n^2) discard pass: the in-bbox filter, the >80%-contained
  shorter-sibling drop, and the empty-input boundary.
@bosd
bosd force-pushed the perf/text-in-bbox-and-random-string branch from 46ad0b8 to 8509555 Compare May 20, 2026 08:38
@bosd
bosd merged commit ed75b68 into camelot-dev:master May 20, 2026
15 of 17 checks passed
@bosd
bosd deleted the perf/text-in-bbox-and-random-string branch May 20, 2026 08:47
bosd added a commit to bosd/camelot that referenced this pull request May 20, 2026
…t-dev#718)

The function that filters textlines to a bbox and discards >80 %-contained
shorter siblings was already cleaned up from O(n^3) to O(n^2) in camelot-dev#718.
Past that, the per-pair Python arithmetic (open-coded intersection,
area, longer-than) dominates. Replace it with NumPy broadcasting:

  1. One pass to pull x0/y0/x1/y1 into parallel float64 arrays.
  2. Vectorised centre-in-bbox filter.
  3. Pairwise NxN intersection area via two maximum/minimum
     broadcasts, divided by per-row 'A' area gives fraction-of-A in B.
  4. b_longer = widths[None,:] > widths[:,None].
  5. discard_any = ((frac>0.8) & b_longer).any(axis=1).

Microbench (random page, single bbox, 20 trials, output bit-identical
to the Python loop on every trial):

    n      Python loop      NumPy        speedup
    50         0.43 ms      0.19 ms      2.3x
   100         1.68 ms      0.63 ms      2.7x
   300        14.62 ms      3.59 ms      4.1x
   800        96.55 ms     28.26 ms      3.4x
  2000       618.35 ms    203.78 ms      3.0x

For inputs larger than 1500 textlines inside the bbox the NxN matrix
would use too much memory (about 32 MB at n=2000) — we fall back to a
side-set Python loop in that case. The fallback retains the early-break
discard semantics from camelot-dev#718. realistic camelot pages cap at a few
hundred textlines per bbox, so the fast path always fires.
bosd added a commit to bosd/camelot that referenced this pull request May 20, 2026
…t-dev#718)

The function that filters textlines to a bbox and discards >80 %-contained
shorter siblings was already cleaned up from O(n^3) to O(n^2) in camelot-dev#718.
Past that, the per-pair Python arithmetic (open-coded intersection,
area, longer-than) dominates. Replace it with NumPy broadcasting:

  1. One pass to pull x0/y0/x1/y1 into parallel float64 arrays.
  2. Vectorised centre-in-bbox filter.
  3. Pairwise NxN intersection area via two maximum/minimum
     broadcasts, divided by per-row 'A' area gives fraction-of-A in B.
  4. b_longer = widths[None,:] > widths[:,None].
  5. discard_any = ((frac>0.8) & b_longer).any(axis=1).

Microbench (random page, single bbox, 20 trials, output bit-identical
to the Python loop on every trial):

    n      Python loop      NumPy        speedup
    50         0.43 ms      0.19 ms      2.3x
   100         1.68 ms      0.63 ms      2.7x
   300        14.62 ms      3.59 ms      4.1x
   800        96.55 ms     28.26 ms      3.4x
  2000       618.35 ms    203.78 ms      3.0x

For inputs larger than 1500 textlines inside the bbox the NxN matrix
would use too much memory (about 32 MB at n=2000) — we fall back to a
side-set Python loop in that case. The fallback retains the early-break
discard semantics from camelot-dev#718. realistic camelot pages cap at a few
hundred textlines per bbox, so the fast path always fires.
bosd added a commit that referenced this pull request May 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant