Repository navigation
🚨 Move processing defaults to class - #49089
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. Pages changed in this PR: |
zucchini-nlp
left a comment
There was a problem hiding this comment.
I really like seeing empty _defaults, let's make it a standard for new releases. We defi need to keep them somewhere, cls attr makes sense to me but prob a private one - my goal is to stop users from exploiting that for unintended things . Quite sure some args in _defaults rn don't make much sense, like the padding=False
As for saving, I don't think saving all call-time kwargs is necessary, imo it goes arg-by-arg and I left a detailed comment in gemma4 👇🏻
LMK what you think
|
run-slow: aria, audioflamingo3, auto, aya_vision, cohere_asr, colpali |
vasqu
left a comment
There was a problem hiding this comment.
Last nits but this is good to go imo!
| self.assertEqual(video_processor_1.size, modified_copied_size_1) | ||
| self.assertEqual(video_processor_1.image_mean, modified_copied_image_mean_1) | ||
|
|
||
| def test_valid_kwargs_set(self): |
There was a problem hiding this comment.
good to keep in mind for potentially audio / feat extractor as well?
There was a problem hiding this comment.
Yes that requires the audio refactor first though.
|
[For maintainers] Suggested jobs to run (before merge) run-slow: align, aria, audioflamingo3, auto, aya_vision, blip, blip_2, bridgetower, bros, canary, chameleon, clipseg, clvp, cohere2_vision, cohere_asr, cohere_compass |
CI (Nvidia + AMD)Nvidia · ✅ No new failures 🎉👏 · ⚙️ Workflow · 📊 Dashboard AMD · ✅ No new failures 🎉👏 · ⚙️ Workflow · 📊 Dashboard Failures (9 pre-existing)Pre-existing failures (still failing) — 9 total: Same error (9): - aria: - aya_vision: |
#49089 and the deepseek_ocr2, mistral_large4 and nemotron3_5_transcribe fixes are on main now; this branch keeps only the mlinter pin and the esm, pp_doclayout_v2 and pp_formulanet TRF055 fixes.
…0740) huggingface#49089 removed it as redundant with `Qwen3ASRFeatureExtractor`, but original checkpoints use `WhisperFeatureExtractor`, which truncates audio longer than 30 s by default. Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
What does this PR do?
ProcessingKwargs._defaultstoProcessorclass attributes following image, video, and audio processors. Defaults are not saved in the config. This will be handled by 🚨🚧 FeatureExtractor → AudioProcessor #44394TextKwargs,ImagesKwargs,VideoKwargsclasses defined inprocessing_*.pyfiles. EmptyProcessingKwargssubclasses are also deprecated.The result are simpler
processing_*.pyfiles with a clearer structure on how to customise behavior.🚨 Breaking
XYZModelProcessorKwargs._defaultsorXYZModelProcessorKwargs.__annotations__return_for_text_completion=Trueis now honouredShieldGemma2Processornow uses the correct processor defaults, previously it used gemma3 defaults by mistake♻️ Deprecated
TextKwargs,ImagesKwargs,VideoKwargssubclasses defined inprocessing_*.pyfilesDeprecatedXYZProcessorKwargsas this routes any imports through__getattr__which can emit the deprecation warning. This is required because those classes are typed dicts and are never instantiated, therefore normal deprecation warnings in__init__are not applicableXYZModelProcessorKwargs.text_kwargs/images_kwargs/video_kwargs/audio_kwargs-> Don't set these anymore as their types are impossible to enforceXYZModelProcessorKwargs._defaults-> Defaults are now specified as class attributes onXYZModelProcessorOther minor changes
common_kwargsdefaults -> Instead set default for each modality explicitly_defaultsthat matched their modality-specific processor defaults were removed_merge_kwargsautomatically merges withself.valid_processor_kwargswhich avoids having to pass them manually (lots of hardcoding)__init__image_seq_lengthin [Gemma3Processor]__call__Noneimages,labels,suffix, etc.MyModelProcessingKwargsclassuse_image_idin [MiniCPMV4_7Processor]Runtime arguments which can have a default must be added to a custom
MyModelProcessingKwargsand will be passed as part of**kwargsto__call__. Defaults for these values can be set as processor class attributes. Defaults for modality-specific processors, for example the tokenizer or image processor, can be defined in the same way. This is usually not needed as modality-specific processors already have defaults defined in their respective classes. Only use this option if the processor must override a modality-specific processor setting. Of course, arguments passed at runtime to__call__have always higher priority than any defaults.How is it enforced?
New tests that check:
valid_processor_kwargsis set to the correct class (same for image and video processors)valid_processor_kwargsare not the same as modality-specific processor default__init__arguments have no default class attribute (avoid multiple defaults)__init__arguments are not invalid_processor_kwargs(avoid multiple defaults)__call__arguments are not invalid_processor_kwargs(different responsibility)__call__arguments are not in class attributes (avoid multiple defaults)__call__arguments have no orNoneas default (avoid multiple defaults)New linter rules that check:
XYZModelProcessorKwargs._defaults/image_kwargs/...: TRF064/065: Add processor kwargs checks transformers-mlinter#87ImagesKwargs/TextKwargs/...inprocessing_*.pyfiles: TRF064/065: Add processor kwargs checks transformers-mlinter#87