[mickqian]

Motivation

potential vram leak is observed with fast-image-processor, due to the memory pool allocated by global cuda allocator held by the tokenizer processor is not released after the image feature is moved to CPU.

Modifications

Use a dedicated MemPool for image processor, scope the processor call with this pool, thus the unused vram will be released and returned to cuda driver once the processor call finishes

Summary

• isolate CUDA allocations from fast image preprocessing in a request-scoped memory pool when fast image processor is enabled and features are immediately transported through CPU
• release the processor scratch-memory high-water mark after feature tensors return to CPU with the lifetime of dedicated memory pool
• preserve existing behavior for CUDA IPC, CUDA VMM, and processors that hash features before CPU transfer

This fixes apparent tokenizer-side VRAM growth during long-running VLM serving. The tensors were already freed, but the process-wide CUDA caching allocator retained the largest preprocessing scratch allocation.

Correctness

Tested with the real BF16 Qwen/Qwen3.8-27B checkpoint on one NVIDIA H200, BF16 KV cache, eager execution, and a seeded 100-item OCRBench sample. The test limit accepted the same 97 requests in every run.

Backend	Score	Time	Tokenizer HBM after run
current main, torchvision fast	73/97	52.59 s	2,412 MiB
current main, PIL	73/97	60.52 s	862 MiB
this PR, torchvision fast	73/97	53.36 s	956 MiB
this PR, second identical replay	73/97	52.14 s	956 MiB

The fast-path answer digest is identical before and after the change: 42af989d32865a9aa3015315116e7670871b0de4788abc9145a3077d6666bb1c. Fast and PIL obtain the same score, so this experiment does not show a model-level accuracy loss from the torchvision processor.

The second replay staying at 956 MiB verifies that processor HBM is bounded rather than growing with repeated requests.

Processor microbenchmark

Six seeded images from 512x512 through 4096x3072, real Qwen3.8 processor, three rounds on NVIDIA H200:

Allocator	Median time	Reserved after run
process-wide default	0.548 s	1,160 MiB
request-scoped pool	0.545 s	2 MiB

Tests

• test/registered/unit/multimodal/test_processor_device_selection.py: 8 passed
• pre-commit hooks: passed

---

CI States

Latest PR Test (Base): ✅ Run #32827919455
Latest PR Test (Extra): ❌ Run #32827919312
Latest PR Test (AMD ROCm 7.2): ❌ Run #32827919291