Repository navigation
Migrate Ghidra import script to PyGhidra with enhanced stack variable… - #1190
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
Summary of ChangesHello @pinksawtooth, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request modernizes the Ghidra import script to align with recent Ghidra versions (12.0+), which utilize PyGhidra and Python 3. The primary goal is to improve the accuracy and detail of string annotations within Ghidra, particularly for stack strings, by commenting not just the function entry but also specific instruction references. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request is a great improvement, migrating the Ghidra script generator to PyGhidra and Python 3. The use of a script template (SCRIPT_TEMPLATE) significantly enhances the maintainability and readability of the generator. The new functionality to find and comment on all references to a stack variable is a major enhancement for reverse engineers using FLOSS.
I've provided a few suggestions to further improve the generated script's quality by addressing an inconsistency in bookmarking, reducing code duplication, and improving type hint consistency. Overall, this is excellent work.
| def append_comment(ea: int, comment_text: str) -> None: | ||
| """ | ||
| Append an EOL comment at the specified address. | ||
|
|
||
| Args: | ||
| ea: The effective address (as integer) | ||
| comment_text: The comment text to append | ||
| """ | ||
| addr = toAddr(ea) | ||
| if addr is None: | ||
| print(f"Warning: Could not resolve address 0x{{ea:x}}") | ||
| return | ||
|
|
||
| cu = currentProgram.getListing().getCodeUnitAt(addr) | ||
| if cu is None: | ||
| print(f"Warning: No code unit at address 0x{{ea:x}}") | ||
| return | ||
|
|
||
| existing_comment = cu.getComment(CodeUnit.EOL_COMMENT) | ||
|
|
||
| if existing_comment is None: | ||
| new_comment = comment_text | ||
| else: | ||
| # Ignore duplicates | ||
| if comment_text in existing_comment: | ||
| return | ||
| new_comment = existing_comment + "\\n" + comment_text | ||
|
|
||
| cu.setComment(CodeUnit.EOL_COMMENT, new_comment) | ||
| createBookmark(addr, "decoded_string", comment_text) | ||
|
|
There was a problem hiding this comment.
The append_comment function has inconsistent bookmarking behavior compared to append_lvar_comment. If a comment already exists, append_comment returns early without creating a bookmark. However, append_lvar_comment (via append_eol_comment_at_address) ensures a bookmark is created even if the comment exists. This can lead to missing bookmarks on subsequent runs.
To ensure consistent behavior, append_comment should also create a bookmark regardless of whether the comment already exists. Moving createBookmark before the check for existing comments would solve this. Since createBookmark is idempotent, this is a safe change.
def append_comment(ea: int, comment_text: str) -> None:
"""
Append an EOL comment at the specified address.
Args:
ea: The effective address (as integer)
comment_text: The comment text to append
"""
addr = toAddr(ea)
if addr is None:
print(f"Warning: Could not resolve address 0x{{ea:x}}")
return
cu = currentProgram.getListing().getCodeUnitAt(addr)
if cu is None:
print(f"Warning: No code unit at address 0x{{ea:x}}")
return
createBookmark(addr, "decoded_string", comment_text)
existing_comment = cu.getComment(CodeUnit.EOL_COMMENT)
if existing_comment is None:
new_comment = comment_text
else:
# Ignore duplicates
if comment_text in existing_comment:
return
new_comment = existing_comment + "\n" + comment_text
cu.setComment(CodeUnit.EOL_COMMENT, new_comment)| def append_eol_comment_at_address(addr, comment_text: str) -> bool: | ||
| """ | ||
| Append an EOL comment at the specified address. | ||
|
|
||
| Returns True if successful, False otherwise. | ||
| """ | ||
| cu = currentProgram.getListing().getCodeUnitAt(addr) | ||
| if cu is None: | ||
| return False | ||
|
|
||
| existing_comment = cu.getComment(CodeUnit.EOL_COMMENT) | ||
|
|
||
| if existing_comment is None: | ||
| new_comment = comment_text | ||
| else: | ||
| if comment_text in existing_comment: | ||
| return True # Already exists | ||
| new_comment = existing_comment + "\\n" + comment_text | ||
|
|
||
| cu.setComment(CodeUnit.EOL_COMMENT, new_comment) | ||
| return True | ||
|
|
||
|
|
||
| def find_stack_var_references(func, target_offset: int): | ||
| """ | ||
| Find all instruction addresses that reference a stack variable at the given offset. | ||
|
|
||
| Args: | ||
| func: The function to search within | ||
| target_offset: The stack frame offset to look for | ||
|
|
||
| Returns: | ||
| List of addresses where the stack variable is referenced | ||
| """ | ||
| ref_addresses = [] | ||
| listing = currentProgram.getListing() | ||
|
|
||
| # Iterate through all instructions in the function | ||
| for inst in listing.getInstructions(func.getBody(), True): | ||
| num_operands = inst.getNumOperands() | ||
| for i in range(num_operands): | ||
| refs = inst.getOperandReferences(i) | ||
| for ref in refs: | ||
| if ref.isStackReference(): | ||
| stack_offset = ref.getStackOffset() | ||
| # Check both positive and negative offsets | ||
| if stack_offset == target_offset or stack_offset == -target_offset: | ||
| ref_addresses.append(inst.getAddress()) | ||
| break | ||
|
|
||
| return ref_addresses | ||
|
|
||
|
|
||
| def append_lvar_comment(fva: int, frame_offset: int, comment_text: str) -> None: | ||
| """ | ||
| Append a comment to a stack variable in Ghidra and to all instructions that reference it. | ||
|
|
||
| Args: | ||
| fva: The function virtual address (as integer) | ||
| frame_offset: The stack frame offset | ||
| comment_text: The comment text to append | ||
| """ | ||
| addr = toAddr(fva) | ||
| if addr is None: | ||
| print(f"Warning: Could not resolve function address 0x{{fva:x}}") | ||
| return | ||
|
|
||
| func = getFunctionContaining(addr) | ||
| if func is None: | ||
| print(f"Warning: No function at address 0x{{fva:x}}") | ||
| return | ||
|
|
||
| commented_count = 0 | ||
|
|
||
| # First, try to add comment to the stack variable itself | ||
| stack_frame = func.getStackFrame() | ||
| if stack_frame is not None: | ||
| for offset in [frame_offset, -frame_offset]: | ||
| var = stack_frame.getVariableContaining(offset) | ||
| if var is not None: | ||
| existing_comment = var.getComment() | ||
| if existing_comment is None: | ||
| new_comment = comment_text | ||
| elif comment_text not in existing_comment: | ||
| new_comment = existing_comment + "\\n" + comment_text | ||
| else: | ||
| new_comment = None # Already exists | ||
| if new_comment is not None: | ||
| var.setComment(new_comment) | ||
| break |
There was a problem hiding this comment.
The generated script template contains duplicated logic for appending comments in append_comment, append_eol_comment_at_address, and append_lvar_comment. This makes the generated script harder to maintain. Consider extracting this logic into a helper function.
For example, you could add a helper function like this:
def _get_updated_comment(existing_comment: str, comment_to_add: str) -> str:
"""
Helper to generate an updated comment string.
Returns the new comment string, or None if no update is needed.
"""
if existing_comment is None:
return comment_to_add
if comment_to_add in existing_comment:
return None # Already exists
return existing_comment + "\n" + comment_to_addAnd then use it in the other functions to simplify them. For example, in append_lvar_comment:
...
var = stack_frame.getVariableContaining(offset)
if var is not None:
new_comment = _get_updated_comment(var.getComment(), comment_text)
if new_comment is not None:
var.setComment(new_comment)
break
...This would improve the quality and maintainability of the generated code.
|
Hey @pinksawtooth, thanks a lot for the contribution! Can you sign the CLA and consider the Gemini provided review comments? |
Bumps [mypy](https://github.com/python/mypy) from 1.16.1 to 1.18.2. - [Changelog](https://github.com/python/mypy/blob/master/CHANGELOG.md) - [Commits](python/mypy@v1.16.1...v1.18.2) --- updated-dependencies: - dependency-name: mypy dependency-version: 1.18.2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [filelock](https://github.com/tox-dev/py-filelock) from 3.18.0 to 3.20.0. - [Release notes](https://github.com/tox-dev/py-filelock/releases) - [Changelog](https://github.com/tox-dev/filelock/blob/main/docs/changelog.rst) - [Commits](tox-dev/filelock@3.18.0...3.20.0) --- updated-dependencies: - dependency-name: filelock dependency-version: 3.20.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [pip](https://github.com/pypa/pip) from 25.1.1 to 25.2. - [Changelog](https://github.com/pypa/pip/blob/main/NEWS.rst) - [Commits](pypa/pip@25.1.1...25.2) --- updated-dependencies: - dependency-name: pip dependency-version: '25.2' dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [black](https://github.com/psf/black) from 25.1.0 to 25.9.0. - [Release notes](https://github.com/psf/black/releases) - [Changelog](https://github.com/psf/black/blob/main/CHANGES.md) - [Commits](psf/black@25.1.0...25.9.0) --- updated-dependencies: - dependency-name: black dependency-version: 25.9.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [pycparser](https://github.com/eliben/pycparser) from 2.22 to 2.23. - [Release notes](https://github.com/eliben/pycparser/releases) - [Changelog](https://github.com/eliben/pycparser/blob/main/CHANGES) - [Commits](eliben/pycparser@release_v2.22...release_v2.23) --- updated-dependencies: - dependency-name: pycparser dependency-version: '2.23' dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [pip](https://github.com/pypa/pip) from 25.2 to 25.3. - [Changelog](https://github.com/pypa/pip/blob/main/NEWS.rst) - [Commits](pypa/pip@25.2...25.3) --- updated-dependencies: - dependency-name: pip dependency-version: '25.3' dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [build](https://github.com/pypa/build) from 1.2.1 to 1.3.0. - [Release notes](https://github.com/pypa/build/releases) - [Changelog](https://github.com/pypa/build/blob/main/CHANGELOG.rst) - [Commits](pypa/build@1.2.1...1.3.0) --- updated-dependencies: - dependency-name: build dependency-version: 1.3.0 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [click](https://github.com/pallets/click) from 8.2.1 to 8.3.0. - [Release notes](https://github.com/pallets/click/releases) - [Changelog](https://github.com/pallets/click/blob/main/CHANGES.rst) - [Commits](pallets/click@8.2.1...8.3.0) --- updated-dependencies: - dependency-name: click dependency-version: 8.3.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
… annotation Updated the script for PyGhidra (Ghidra 12.0+) compatibility, migrating from Jython (Python 2.7) to Python 3. The main improvement is enhanced local variable commenting that now annotates both the stack variable definition and all instruction references to it. Key changes: - Add PyGhidra compatibility for Ghidra 12.0+ - Implement find_stack_var_references() to track all references to stack variables - Update append_lvar_comment() to add EOL comments at reference sites - Refactor using template-based code generation for better maintainability - Add type hints and comprehensive docstrings - Support both Pydantic v1 and v2 for result parsing This makes the generated script more effective at annotating stack strings by showing decoded strings at the exact instructions where they are referenced, rather than just at the function entry point. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
Please ensure black and other lints pass. Do you have feedback on the code review comments? |
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
black still fails |
#1190) * Migrate Ghidra import script to PyGhidra with enhanced stack variable annotation Updated the script for PyGhidra (Ghidra 12.0+) compatibility, migrating from Jython (Python 2.7) to Python 3. The main improvement is enhanced local variable commenting that now annotates both the stack variable definition and all instruction references to it. Key changes: - Add PyGhidra compatibility for Ghidra 12.0+ - Implement find_stack_var_references() to track all references to stack variables - Update append_lvar_comment() to add EOL comments at reference sites - Refactor using template-based code generation for better maintainability - Add type hints and comprehensive docstrings - Support both Pydantic v1 and v2 for result parsing This makes the generated script more effective at annotating stack strings by showing decoded strings at the exact instructions where they are referenced, rather than just at the function entry point. 🤖 Generated with [Claude Code](https://claude.com/claude-code) * Update scripts/render-ghidra-import-script.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * style: format ghidra import script with black --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
mandiant#1190) * Migrate Ghidra import script to PyGhidra with enhanced stack variable annotation Updated the script for PyGhidra (Ghidra 12.0+) compatibility, migrating from Jython (Python 2.7) to Python 3. The main improvement is enhanced local variable commenting that now annotates both the stack variable definition and all instruction references to it. Key changes: - Add PyGhidra compatibility for Ghidra 12.0+ - Implement find_stack_var_references() to track all references to stack variables - Update append_lvar_comment() to add EOL comments at reference sites - Refactor using template-based code generation for better maintainability - Add type hints and comprehensive docstrings - Support both Pydantic v1 and v2 for result parsing This makes the generated script more effective at annotating stack strings by showing decoded strings at the exact instructions where they are referenced, rather than just at the function entry point. 🤖 Generated with [Claude Code](https://claude.com/claude-code) * Update scripts/render-ghidra-import-script.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * style: format ghidra import script with black --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Updated the script for PyGhidra (Ghidra 12.0+) compatibility, migrating from Jython (Python 2.7) to Python 3. The main improvement is enhanced local variable commenting that now annotates both the stack variable definition and all instruction references to it.
Key changes:
This makes the generated script more effective at annotating stack strings by showing decoded strings at the exact instructions where they are referenced, rather than just at the function entry point.