Skip to content

TRF061: ProcessingKwargs modality fields must use the shared kwargs types. - #79

Closed
guarin wants to merge 1 commit into
huggingface:mainfrom
guarin:check-processing-kwargs
Closed

guarin wants to merge 1 commit into
huggingface:mainfrom
guarin:check-processing-kwargs

Conversation

@guarin

@guarin guarin commented Oct 13, 2026

Copy link
Copy Markdown
Member

Follow up from huggingface/transformers#49089, in the future we want to have this:

class MyModelProcessorKwargs(ProcessingKwargs):
     images_kwargs: ImagesKwargs

instead of:

class MyModelProcessorKwargs(ProcessingKwargs):
     images_kwargs: MyModelImagesKwargs

@guarin
guarin marked this pull request as draft October 21, 2026 09:33
@guarin guarin closed this Oct 27, 2026
tarekziade added a commit that referenced this pull request Oct 28, 2026
## Why


[huggingface/transformers#49748](huggingface/transformers#49748):
`from transformers import *` crashes on an install with Pillow but no
torch. `Gemma4Processor` / `Gemma4UnifiedProcessor` are
`@requires(backends=("vision",))`, but under `if is_vision_available():`
they import `.image_processing_gemma4[_unified]`, which does `import
torch` / torchvision at top level. The lazy module imports the processor
file as soon as vision is installed, and the sibling import crashes.

TRF009 doesn't catch this because it only bans imports from *other*
models; imports within a model folder are fine. CI didn't catch it
either, because the runtime "PIL only" import step was running without
PIL (fixed in huggingface/transformers-ci#341). This rule catches the
pattern at lint time.

## What TRF064 checks

For every relative import that runs at module import time:

- **Declared** backends = the file's base requirements (mirror of
`BASE_FILE_REQUIREMENTS` in transformers' `import_utils.py`) βˆͺ the
*weakest* `@requires(backends=...)` among its `__all__` objects, since
the lazy loader imports the file as soon as any one exported object's
backends are present.
- **Needed** backends = what the imported model-local files' unguarded
top-level imports pull in (`import torch` β†’ torch, `from PIL` β†’ vision,
…), followed transitively, with implied backends (torchvision β‡’ torch).
- Flags when needed βŠ„ declared βˆͺ enclosing `is_<backend>_available()`
guards.
- Skips function/class bodies, `TYPE_CHECKING`, and `try`/`except
ImportError`. Core deps (`tokenizers`, `numpy`, …) are never backends.
- `modular_*.py` files are checked through the files they generate,
since generated files aren't linted. The violation anchors on the
matching import or exported class in the modular file, so `# trf-ignore:
TRF064` works there.
- The message shows the full import chain.

The lint cache now hashes every sibling file in the model directory, so
editing an imported file re-checks its importers.

## Results on transformers

| Tree | TRF064 hits |
|---|---|
| `main` @ `fb3e4f2828` | 2: `gemma4/processing_gemma4.py:L28` and
`gemma4_unified/modular_gemma4_unified.py:L340` |
| PR #49748 head | 0 |

These are exactly the two classes that fail `from transformers import *`
at runtime in the `huggingface/transformers-consistency` image with only
Pillow installed. There are no other hits (an earlier draft treated
`tokenizers` as optional and produced 3 false positives; it is a core
dependency).

`make lint`, `make test` (454 passed) and `make typecheck` all pass.

## Limitations / follow-ups

- Base requirements and the package β†’ backend map are hand-mirrored from
transformers and can drift.
- Direct third-party imports (e.g. top-level `import torch` in a
`processing_*.py` with only `@requires("vision")`) are not flagged yet:
the rule is scoped to model-local imports to keep this first version
precise. Worth a follow-up after a false-positive pass.
- Adopting it in transformers needs a TRF064 entry in
`utils/rules.toml`.
- TRF060–063 are taken by open PRs (#77, #79, #81, #82, #85), hence 064.

πŸ€– Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant