Repository navigation
Releases
BE v2.2.0
What's Changed
- Update continuous-integration-pip.yml by @simsong in #479
- Update identify_filenames.py by @h-ssh in #482
- Update LICENSE.md by @simsong in #478
- add missing include by @rick-gnous in #492
- Patch maint by @simsong in #495
- New vinscan by @simsong in #494
- Integrate be20 into the bulk_extractor source tree by @simsong in #498
- Fix sbuf bounds contract by @simsong in #511
- Stop notifier on phase-one write failures by @simsong-codex in #513
- Make E01 and split-image input selection safe by @simsong-codex in #517
- Repair packet address parsing and bounds checks by @simsong-codex in #516
- Run AddressSanitizer once per pull request by @simsong-codex in #514
- Mark 2.2.0 development builds by @simsong-codex in #520
- Let the image reader clean up on disk errors by @simsong-codex in #519
- Handle short image reads by @simsong-codex in #518
- Cover explicit Outlook scanner selection by @simsong-codex in #527
- Honor jpeg_carve_mode=0 by @simsong-codex in #525
- Repair sbuf file mapping ownership by @simsong-codex in #524
- Update debt tracker after #420 resolution by @simsong-codex in #526
- Bound fallback PCAP reads by @simsong-codex in #530
- Document the scanner API lifecycle by @simsong-codex in #529
- Fix feature recorder histogram contract by @simsong-codex in #531
- Propagate banner files to feature recorders by @simsong-codex in #533
- Preserve features after histogram allocation failures by @simsong-codex in #535
- Restore loadable scanner modules by @simsong-codex in #528
- Fix optional Exiv2 configuration by @simsong-codex in #532
- Add recent work report and management briefing by @simsong-codex in #536
- Fix MSXML sbuf leak handling by @simsong in #537
- Document Codex GitHub identity by @simsong-codex in #542
- Limit ZIP carved filenames by @simsong in #539
- Make recursive directory traversal safe by @simsong in #541
- Add consolidated 2.2.0 and historical release notes by @simsong-codex in #544
- Add Ubuntu MinGW Windows artifact build by @simsong-codex in #543
- Fix code scanning alerts by @simsong in #538
- Honor --max_minute_wait by @simsong in #540
- Move technical debt scoreboard to GitHub Discussion by @simsong-codex in #546
- Test recursive Unicode filenames on Windows by @simsong-codex in #549
- Add community contribution guidelines by @simsong-codex in #551
- Restore alert-list processing by @simsong-codex in #553
- Add RawTherapee Image8 thumbnail carver by @simsong-codex in #556
- Restore stop-list processing by @simsong-codex in #552
- Fix IPv6 packet checksum validation by @simsong-codex in #554
- Publish current 2.2 LaTeX documentation on Pages by @simsong-codex in #557
- Test -e all scanner selection by @simsong-codex in #562
- Refresh 2.2.0 release notes by @simsong-codex in #560
- Fix #515: preserve 802.11 PCAP frames by @simsong-codex in #559
- Expand the scanner development manual by @simsong-codex in #558
- Replace generic INSTALL instructions by @simsong-codex in #561
- Test builds in Debian Bookworm by @simsong-codex in #565
- Exclude HTML quote entities from URLs by @simsong in #568
- Support big-endian utmp records by @simsong-codex in #563
- Stabilize Debian CI by gating optional-dependency test paths by @simsong with @Copilot in #567
- Document Debian CI scope by @simsong-codex in #572
- Fix Windows IP address formatting by @simsong-codex in #573
- Warn when built without O3 optimization by @simsong-codex in #564
- Clean up C++ initialization warnings by @simsong-codex in #571
- Tighten Windows IP regression assertion by @simsong-codex in #574
- Quote xmllint test paths by @simsong-codex in #577
- Refresh release notes for recent closures by @simsong-codex in #579
- Bypass duplicate sbuf scanner fan-out by @simsong-codex in #578
- Send O3 warning to diagnostic stream by @simsong-codex in #576
- Remove disabled scanner-set test by @simsong-codex in #580
- Use zip_carved output directory by @simsong-codex in #582
- Test context-sensitive stop lists by @simsong-codex in #583
- Add diagnostic logging foundation by @simsong-codex in #575
- Remove nested output directories with -Z by @simsong-codex in #581
- Harden duplicate-sbuf diagnostics by @simsong-codex in #584
- Honor JPEG recorder carve limits by @simsong-codex in #590
- Document review policy and ignore generated build outputs by @simsong-codex in #589
- Test Word PDF email extraction by @simsong-codex in #591
- Add case-sensitive find option by @simsong-codex in #587
- Support Windows raw-device input by @simsong-codex in #588
- Bound extracted email local parts and domains independently by @simsong-codex in #593
- Test scanner configuration find-option isolation by @simsong-codex in #592
- Adapt progress display to Windows resize by @simsong-codex in #595
- Recover all members from RAR archives by @simsong-codex in #594
- Document carving mode behavior by @simsong-codex in #596
- Validate report DFXML against the bundled schema by @simsong-codex in #598
- Verify Copilot review requests by @simsong-codex in #600
- Retire obsolete numeric debug mask by @simsong-codex in #599
- Test and repair interrupted-run restart by @simsong-codex in #597
- Fix 2026 Coverity parser findings by @simsong in #605
- Initialize scoped test environment state by @simsong-codex in #603
- Add container image for regular scans by @simsong-codex in #607
- Harden truncated hibernation headers by @simsong-codex in #608
- Accept current email TLDs by @simsong-codex in #604
- Remove stale Lightgrep Base16 scanner by @simsong-codex in #609
- Report restart recovery status by @simsong-codex in #610
- Bound RAR PPM dictionary copies by @simsong-codex in #602
- Add ThreadSanitizer thread-pool coverage by @simsong in #606
- Document Copilot fix thread replies by @simsong-code...
BE v2.1.1
BE v2.1.0
January 22, 2024
RELEASE NOTES
The digital forensics tool bulk_extractor version 2.1.0 is now available for general use.
Release download point:
https://github.com/simsong/bulk_extractor/releases
GIT repository:
https://github.com/simsong/bulk_extractor
I am pleased to announce the general availability of bulk_extractor version 2.1. This is the first release of bulk_extractor version 2 that is recommended for general use.
Bulk_extractor 2 is a significant rewrite of bulk_extractor. Verison 2 significantly improves the performance and portability of version 1. The rewrite started in 2016 and was largely completed by January 2021.
Details of the rewrite, including a detailed report of the performance improvements and lessons learned, can be found in Sharpening Your Tools: Updating bulk_extractor for the 2020s, Simson Garfinkel and Jon Stewart. Communications of the ACM, August 2023.
Bulk_extractor version 2.1 is the first stable version of bulk_extractor version 2 that is recommended for general use. It corrects a problem with the string search scanner that caused bulk_extractor to hang on open-ended regular expressions such as [a-z]*@company.com specified with the -F flag. With version 2.1, we have replaced the C++17 regex compiler with Google's RE2 regex compiler that avoids backtracking. As a result, these open-ended regular expressions no longer hang.
2.0/2.1 Improvements over Version 1:
- BE2 is significantly faster on multi-core systems than BE1.
Release 2.1 Limitations
-
BEViewer is not included in this release. Although it works with Version 2, it is not yet officially supported.
-
scan_outlook and scan_hiberfile are now disabled by default because they did not have unit tests. These scanners can be re-enabled by specifying -eoutlook and -ehiberfile on the command line.
-
scan_aes no longer scans for 192-bit AES keys by default, although this behavior can be re-enabled.
Known bugs:
-
The RAR decompressor does not reliably decompress all RAR files and only supports RAR v1, v2, and v3.
-
The RAR scanner will not reliably name carved RAR file components that contain UTF-8 characters in their name.
You can help
We are looking for help to implement the following algorithms:
-
WkdmDecompress - http://www.opensource.apple.com/source/xnu/xnu-1456.1.26/iokit/Kernel/WKdmDecompress.c
-
xz, 7zip, and LZMA/LZMA2 decompression
-
lzo decompression
-
BZIP2 decompression
-
CAB decompression
-
Scanning for the start of BitLocker protected volumes.
-
NTFS decompression
-
Better handling of MIME encoding
-
Process more data with -e xor and look for CCN hits. Most will be false positives
-
Demonstration of bulk_extractor running on a grid (how fast can it run?)
-
Python Bridge - run multiple copies of python to let scanners be written in python
-
scan_pipe - runs every sbuf through an external program.
BE v2.0.6
BE v2.0.3
BE v2.0.0
Release 2.0.0 of bulk_extractor, a high-performance digital forensics tool that works like a "find evidence" button, pulling actionable intelligence out of disk images, files, memory dumps, network traffic, and just about anything else.
Note: we recommend using the bulk_extractor-2.0.0.tar.gz file attached, which is a proper release, rather than cloning the repo and all of the sub-repos and then using automake to create the configure script.
BE v1.5.3
BE v1.4.0
BE v1.4.0-beta1
Please let us know how it works. We are especially interested in feedback on the XOR scanner.