[TarangKhanna]

Motivation

Weight cache daemon discovery currently uses distributed rank in the Unix socket and ready file paths. Rank is only unique within one distributed job.

Two independent jobs on the same host can therefore both have rank 0 while running on different physical GPUs, allowing one job to discover the other job's daemon.

Change

Use the physical GPU UUID for daemon socket and ready file identity instead of distributed rank.

The existing launcher topology is unchanged. Daemons are still placed using the existing PP and TP local GPU mapping. When cleanup or readiness needs the daemon path, the launcher derives the UUID from that same local GPU id.

The client follows the same rule. If no explicit weight_cache_socket is configured, IpcModelLoader derives the daemon socket from the UUID of the GPU it is actually loading onto.

Weight compatibility remains separate from discovery. After connecting, the existing CacheConfig check still validates the model and shard configuration, including TP, PP, DP, EP, quantization, dtype, and revision.

Scope

This change is limited to node local daemon discovery identity.

Validation

1. test/registered/unit/model_loader/test_weight_cache_protocol.py: 31 passed.

2. Unit coverage verifies that different physical GPU UUIDs produce different socket and ready paths.

3. Unit coverage verifies that automatic discovery uses the caller's actual GPU id rather than logical rank or an implicit GPU 0.

4. Existing static DP and EP compatibility coverage remains passing.

5. TP1 CUDA end to end validation passed on A40.

6. TP2 CUDA end to end validation passed on two H100 SXM GPUs, including daemon startup, automatic discovery, CUDA IPC weight import, and generation.

---

CI States

Latest PR Test (Base): ⏳ Run #33374336440
Latest PR Test (Extra): ⏳ Run #33374336245
Latest PR Test (AMD ROCm 7.2): ⏳ Run #33374336462

[liusy58]

@TarangKhanna Is this PR ready for review yet? Can we limit the scope of the codebase it modifies?

[TarangKhanna]

@TarangKhanna Is this PR ready for review yet? Can we limit the scope of the codebase it modifies?

@liusy58 Not quite yet. I’m doing one final pass to reduce the diff. The change is broader than just changing the socket key because moving from rank based identity to GPU UUID plus config also means the old rank based ready file can no longer provide the ownership/readiness coordination we need. The new path uses an identity scoped lock for ownership and the socket itself for readiness. I’m trimming the remaining lifecycle and wiring changes to only what’s required to preserve those, then I’ll mark it ready once the diff is tighter.

[liusy58]

@liusy58 Not quite yet. I’m doing one final pass to reduce the diff. The change is broader than just changing the socket key because moving from rank based identity to GPU UUID plus config also means the old rank based ready file can no longer provide the ownership/readiness coordination we need. The new path uses an identity scoped lock for ownership and the socket itself for readiness. I’m trimming the remaining lifecycle and wiring changes to only what’s required to preserve those, then I’ll mark it ready once the diff is tighter.

Thanks for your contribution. We’d like to keep the scope of the changes as tight as possible.

[TarangKhanna]

@liusy58 Not quite yet. I’m doing one final pass to reduce the diff. The change is broader than just changing the socket key because moving from rank based identity to GPU UUID plus config also means the old rank based ready file can no longer provide the ownership/readiness coordination we need. The new path uses an identity scoped lock for ownership and the socket itself for readiness. I’m trimming the remaining lifecycle and wiring changes to only what’s required to preserve those, then I’ll mark it ready once the diff is tighter.

Thanks for your contribution. We’d like to keep the scope of the changes as tight as possible.

I agree. I’ll trim this back to the discovery change and keep the existing startup behavior where possible. The remaining changes will focus on identifying daemons by physical GPU UUID plus cache config, with only the minimal loader wiring needed for that.