[yaoyu-33]

What does this PR do?

Use the standard Megatron cyclic dataloader for Direct-HF SFT shuffling instead of mutating loaded examples inside the dataset builder.

Direct-HF SFT configs now default to dataloader_type="cyclic". The in-tree VLM/audio SFT and PEFT recipes that previously pinned the sequential single sampler now explicitly select cyclic, and the MegatronMIMO Qwen3.5-VL example uses the same CLI default.

Why cyclic instead of batch?

cyclic selects MegatronPretrainingRandomSampler, which provides deterministic epoch shuffling and resume-aware sample ordering while keeping DataLoader collation at microbatch granularity. That preserves the existing VLM in-batch packing behavior.

The batch sampler collates a rank-local global batch before the finetuning step splits it into microbatches. For Direct-HF VLM recipes this can change the packing scope and can be incompatible with collators that pack the input down to a batch dimension of one.

Changelog

• Default DirectHFSFTDatasetConfig.dataloader_type to cyclic.
• Use cyclic in Direct-HF SFT/PEFT recipes for common VLM, Nemotron Omni, Qwen2-Audio, and Qwen3-Omni.
• Default the MegatronMIMO Qwen3.5-VL Direct-HF example to cyclic.
• Remove the builder-level shuffle flags, dedicated seed, and list mutation introduced by the first revision of this PR.
• Add config, recipe, serialization, and example-default assertions.

Users can still request source-order sampling explicitly with dataloader_type="single".

Validation

• uv run --isolated --no-project --with pre-commit==4.6.0 pre-commit run --all-files — passed.
• uv run --isolated --no-project python -m compileall -q <changed Python files> — passed.
• git diff --check — passed.
• Targeted pytest collection is pending Linux CI. On the local macOS ARM environment, uv run is blocked before pytest by the Linux-only nvidia-resiliency-ext wheel; the current Megatron-Core import also requires Triton.

Before your PR is Ready for review

• Read and followed the contributor guidelines.
• Added unit-level contract coverage.
• No new dependency or public conversion API change.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[claude]

Light review - LGTM

Deterministic shuffling for Direct-HF SFT splits with a clean opt-out and no source mutation. Logic is correct: list(examples) copies before shuffling so the loaded source is preserved, and random.Random(seed) keeps it deterministic and isolated from global RNG. Tests cover default-shuffle-without-mutation, the shuffle=False opt-out, and round-trip serialization of the two new config fields.

Minor observations (non-blocking):

• All three splits (train/validation/test) share shuffle_seed. Each split is a separate list and shuffling happens before truncation to target_length, so the selected subset stays deterministic across runs, which looks intended.
• Consider noting in the docstring/config field that shuffling also applies to validation/test, not just train.

Suggested test cases

• tests/unit_tests/data/builders/test_direct_hf_sft.py::test_builder_shuffles_examples_by_default_without_mutating_source
• tests/unit_tests/data/builders/test_direct_hf_sft.py::test_builder_allows_source_order_opt_out
• tests/unit_tests/data/builders/test_direct_hf_sft.py::test_direct_hf_sft_config_round_trip_is_declarative
• (gap) A case asserting validation/test splits are also shuffled (current tests set do_validation=False/do_test=False, so the shuffle path is only exercised for train).
• No perf tests impacted.

[copy-pr-bot]

/ok to test 071fc4afa8444ed2f3d4542181321794329bbf6d

@yaoyu-33, there was an error processing your request: E2

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/2/

[yaoyu-33]

/ok to test 071fc4a