[rollout, perf] aggressive pre-resume cache cleanup regresses colocated weight sync #7848

Summary

After upgrading veRL v0.8.0 → v0.9.0, we observed a colocated GRPO regression on 8×H200/Qwen3-8B (V0 on both releases, identical Hydra overrides):

v0.9 added aggressive_empty_cache(force_sync=True) immediately before resuming rollout weights. Removing only that call from v0.9 saved 1.133 s (about 46% of the release gap); adding it to v0.8 cost 0.695 s. The release locks also differ, so the call is a material contributor, not the entire regression.

Why cleanup exists

PyTorch and vLLM are separate colocated processes. PyTorch must return unused reserved allocator blocks before vLLM remaps weights. empty_cache() performs that handoff without freeing live tensors.

The aggressive helper also runs full Python GC, synchronizes, and retries after releasing ≥1 GiB. PR #7242 added this call in an unrelated DeepSeek-V4 change without explaining it; its placement suggests conservative memory-headroom protection.

Native-V1 A/B (veRL v0.9.0)

Medians exclude cold step 1 and deliberate calibration steps 3–4. All six runs completed without OOM. Phase traces show:

• First empty_cache() released 7.66–40.55 GiB/rank and remains necessary.
• GC released zero CUDA bytes but cost 0.56–0.69 s.
• Retry triggered on every measured rank/step, released zero additional cache, and cost another 0.55–0.63 s.
• Explicit synchronization after empty_cache() was <0.8 ms.

Synthetic tests confirm GC can help when unreachable Python cycles own CUDA tensors; that condition was not observed in these natural traces.

Proposed change

- aggressive_empty_cache(force_sync=True)
+ get_torch_device().empty_cache()
  await self.rollout.resume(tags=["weights"])

Change only this pre-resume call. Keep the separate post-transfer aggressive cleanup before KV-cache resume. PR #7780 changes sender cleanup at another boundary and is not a duplicate; no overlapping open PR was found.