Repository navigation
Snippets: split parse_snippets into overridable steps - #3040
Conversation
|
While splitting out path = '' if m is None else m.group(1).strip()
# Looks like we have an empty file and only lines specified
if not path:
if self.check_paths:
raise SnippetMissingError(f"Snippet at path '{path}' could not be found")
It doesn't say which include caused it. To keep this PR a pure refactor, I left the message exactly as it was. After the refactor the raw include string ( |
|
This is likely not exactly what people envisioned when they desired a way to override things, as they weren't coming from the angle of subclassing the extension, but I'm not against refactoring Snippets to allow exactly that, especially if it introduces no adverse effects or performance issues. To be honest, if we are going to do a refactor, we should audit performance issues now, before other people with AI find them and force us to refactor yet again, breaking some of your expectations. It may already be fine, though, but I haven't checked.
Sure, better errors make sense. |
|
I'm actually going to perform an audit on Snippets. You might have to rebase when I'm done identifying issues. |
|
@facelessuser I'm glad you are considering merging this :) I will keep an eye out for any updates! |
|
Small refactor to subclassing doesn't cost me much, but my biggest focus is on shutting down the endless AI issues/PR about: quadratic behavior, recursion error, etc. This is a drain on me right now across multiple repositories I maintain, and I am aggressively going after those types of things, before anything else. It's getting less on a number of projects, but the focus usually shifts to another project. |
Extract parse_snippet_target, extract_lines, select_snippet_lines and a no-op process_snippet_lines hook so subclasses can extend the include syntax and post-process snippet content. No behavior change. Rebased onto the Snippets performance work: parse_snippet_target uses the linear split_path (not the RE_SNIPPET_FILE regular expression) so long paths with many line ranges stay linear. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Build on the overridable steps in the previous commit. `process_snippet_lines` runs before nested snippets are expanded, so the lines it receives still contain snippet notation. A text rewrite, such as a find and replace, can change the notation of a nested snippet and make it quietly disappear (or fail with a confusing path when `check_paths` is enabled). Keep that ordering, as it matches how line and section selection already work, and make the contract explicit: - Spell out the ordering and its consequences in the `process_snippet_lines` docstring, including that nested snippets are not seen by the hook. - Add `get_syntax_mask`, which returns a boolean per line that is `True` for snippet notation (inline snippets, block markers and the paths between them, and section markers). Escaped notation is content. It mirrors the loop in `parse_snippets` so hooks can leave notation alone. - Add an "Extending Snippets" section to the docs describing each step, the target keys, the ordering warning, and a complete example that renames a term without touching notation. - Add a changelog entry. Tests cover the mask directly, check it against what `parse_snippets` actually keeps as content on randomly generated lines, and use the documented example to show both that leaving notation alone keeps nested snippets working and that an unprotected rewrite loses them. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
a5aa730 to
5a4a671
Compare
|
@joapuiib I've made a number of performance-related changes and rebased your stuff on top. A couple of additional changes were made on top of that. Please review the changes. If you have concerns or want to make more changes to fit your needs, please commit those, and we can discuss them. |
|
@facelessuser LGTM I've tested it with my custom extension providing a |
|
@joapuiib If you can add |
|
@gir-bot lgtm |
This PR refactors
SnippetPreprocessor.parse_snippetsinto a few smaller methods, so subclasses can extend the snippet syntax and post-process the content without copying the whole parsing loop.This is a pure refactor: the snippet syntax, the output and the error messages stay exactly as before.
Changes
parse_snippet_target(path): parsesfile.md:1:5,file.md:1:2,8:9orfile.md:sectioninto a dict withpath,start,endandsection. ReturnsNoneif no path is found.extract_lines(...)/select_snippet_lines(lines, target): pick the requested lines or section from the loaded snippet.process_snippet_lines(lines, target): a hook that does nothing by default. It runs after line selection and before nested snippets are parsed.Existing tests pass unchanged, and new tests cover the extracted methods and a subclass overriding the hooks.
Motivation
I'd like to build a third-party extension on top of snippets that transforms included content with chained filters. For example, it could turn an abbreviations file into a glossary list:
With this PR the whole extension comes down to two overrides:
How this relates to existing issues and PRs
process_snippet_lines(lines, target)is the post-process hook. It answers the "more context" concern:targetcarries the snippet path or URL,is_url, and the lines or section requested. Subclasses also haveself.md.parse_snippet_targetto add syntax in a third-party extension. Core syntax and behaviour stay the same, which fits Snippets' goal of "easy, dumb, and predictable inserts".Note
This PR (code, tests and description) was prepared with the help of an LLM. I have reviewed and checked all of it before submitting.