Repository navigation
perf(utils): speed up text_in_bbox (≈10x) and random_string (4x) - #718
Merged
bosd merged 3 commits intoMay 20, 2026
Merged
Conversation
This was referenced May 20, 2026
Two low-hanging hotspots identified during a profiling pass: * random_string used per-char string concatenation in a while-loop and rebuilt the alphabet on every iteration. Switched to random.choices + str.join with the alphabet cached at import time. Microbench: 4.0x faster (10k calls of length 32: 203ms → 51ms). * text_in_bbox ran an O(n³) discard pass because rest.copy() was called inside the outer loop. The same set membership semantics can be expressed with a side 'discarded' set of indices and no copies. Microbench on 800 synthetic boxes: 9.5x faster (165ms → 17ms), with identical kept-set. Both changes preserve observable behaviour (rendered table content is unchanged); only allocation/iteration patterns differ. See the perf report on the PR for the broader profiling investigation.
Adds two unit tests that exercise the refactored hot paths directly: - test_random_string_length_and_alphabet covers the new random.choices + str.join path, including the length=0 edge case that the previous while-loop hit by accident (loop body skipped). - test_text_in_bbox_filters_and_discards_overlaps covers the new O(n^2) discard pass: the in-bbox filter, the >80%-contained shorter-sibling drop, and the empty-input boundary.
bosd
force-pushed
the
perf/text-in-bbox-and-random-string
branch
from
May 20, 2026 08:38
46ad0b8 to
8509555
Compare
bosd
added a commit
to bosd/camelot
that referenced
this pull request
May 20, 2026
…t-dev#718) The function that filters textlines to a bbox and discards >80 %-contained shorter siblings was already cleaned up from O(n^3) to O(n^2) in camelot-dev#718. Past that, the per-pair Python arithmetic (open-coded intersection, area, longer-than) dominates. Replace it with NumPy broadcasting: 1. One pass to pull x0/y0/x1/y1 into parallel float64 arrays. 2. Vectorised centre-in-bbox filter. 3. Pairwise NxN intersection area via two maximum/minimum broadcasts, divided by per-row 'A' area gives fraction-of-A in B. 4. b_longer = widths[None,:] > widths[:,None]. 5. discard_any = ((frac>0.8) & b_longer).any(axis=1). Microbench (random page, single bbox, 20 trials, output bit-identical to the Python loop on every trial): n Python loop NumPy speedup 50 0.43 ms 0.19 ms 2.3x 100 1.68 ms 0.63 ms 2.7x 300 14.62 ms 3.59 ms 4.1x 800 96.55 ms 28.26 ms 3.4x 2000 618.35 ms 203.78 ms 3.0x For inputs larger than 1500 textlines inside the bbox the NxN matrix would use too much memory (about 32 MB at n=2000) — we fall back to a side-set Python loop in that case. The fallback retains the early-break discard semantics from camelot-dev#718. realistic camelot pages cap at a few hundred textlines per bbox, so the fast path always fires.
bosd
added a commit
to bosd/camelot
that referenced
this pull request
May 20, 2026
…t-dev#718) The function that filters textlines to a bbox and discards >80 %-contained shorter siblings was already cleaned up from O(n^3) to O(n^2) in camelot-dev#718. Past that, the per-pair Python arithmetic (open-coded intersection, area, longer-than) dominates. Replace it with NumPy broadcasting: 1. One pass to pull x0/y0/x1/y1 into parallel float64 arrays. 2. Vectorised centre-in-bbox filter. 3. Pairwise NxN intersection area via two maximum/minimum broadcasts, divided by per-row 'A' area gives fraction-of-A in B. 4. b_longer = widths[None,:] > widths[:,None]. 5. discard_any = ((frac>0.8) & b_longer).any(axis=1). Microbench (random page, single bbox, 20 trials, output bit-identical to the Python loop on every trial): n Python loop NumPy speedup 50 0.43 ms 0.19 ms 2.3x 100 1.68 ms 0.63 ms 2.7x 300 14.62 ms 3.59 ms 4.1x 800 96.55 ms 28.26 ms 3.4x 2000 618.35 ms 203.78 ms 3.0x For inputs larger than 1500 textlines inside the bbox the NxN matrix would use too much memory (about 32 MB at n=2000) — we fall back to a side-set Python loop in that case. The fallback retains the early-break discard semantics from camelot-dev#718. realistic camelot pages cap at a few hundred textlines per bbox, so the fast path always fires.
bosd
added a commit
that referenced
this pull request
May 20, 2026
This was referenced May 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Low-hanging fruit from a profiling pass on
camelot/utils.py. Both are pure refactors — output is bit-for-bit identical to the previous implementation.text_in_bbox: O(n³) → O(n²), ≈9.5× fasterThe 'discard >80%-contained boxes' phase called
rest.copy()inside the outer loop, making it O(n³) in the size oft_bbox. Tracked discards in a sideset[int]of indices instead; same semantics, no per-iteration copies.Microbench (800 synthetic boxes, 1000-unit page):
This function is called once per (table_area × parse), and
nis the count of text objects on a page — typically hundreds, sometimes thousands on dense reports. Expect a noticeable end-to-end win on those PDFs; smaller pages will not regress.random_string: ≈4× faster, no behavioural changerandom.choicein a Python loop withret += …rebuilds the alphabet on every iteration and allocates a fresh string each step. Replaced with onerandom.choices(_AL, k=length)+str.join, alphabet cached at module import.Microbench (10 000 calls of length 32):