[dingqingy-nv]

Re-enable virtual pipeline parallelism for Qwen3 235B A22B: VP=12 for GB300 MXFP8, VP=3 for GB200 MXFP8.

locally checked gb200 nan grad is gone. confirming with CI for other cases.

Summary by CodeRabbit

• Refactor
  • Updated model training configurations for GPU-based workloads by adjusting virtual pipeline parallelization settings. Changes optimize resource distribution across different hardware system configurations to improve training performance.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 05573d34-14c9-4ed9-adb0-68adfe6e129b

📥 Commits

Reviewing files that changed from the base of the PR and between f689296 and 2e9caa8.

📒 Files selected for processing (1)

• scripts/performance/configs/qwen/qwen3_workload_base_configs.py

---

📝 Walkthrough

Walkthrough

Two MX (mixed precision) workload base configurations for QWEN3 are redefined from direct aliases to derived configurations using replace(), adjusting the virtual_pipeline_model_parallel_size parameter while preserving other settings from their CS_V2 base configurations.

Changes

Cohort / File(s)	Summary
Configuration Derivation Updates scripts/performance/configs/qwen/qwen3_workload_base_configs.py	Two MX_V2 configuration variables changed from aliases to replace()-based derivations with adjusted virtual_pipeline_model_parallel_size values (12 for GB300, 3 for GB200).

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name	Status	Explanation	Resolution
Test Results For Major Changes	⚠️ Warning	PR modifies virtual pipeline parallelism settings affecting training numerics and performance, but PR description lacks test results, performance metrics, and comprehensive validation evidence for both configurations.	Add links to CI results, convergence plots, performance metrics for both GB300/GB200 configs, and explicit confirmation that NaN gradient issue is resolved across all affected configurations.
Title check	❓ Inconclusive	The title 'NaN grad gone, re-enable VP' is vague and uses informal terminology that doesn't clearly convey the specific changes. While it relates to the PR objective of re-enabling virtual pipeline parallelism, it lacks clarity about the specific configurations being modified or the technical nature of the fix.	Consider revising to a more descriptive title like 'Re-enable virtual pipeline parallelism for Qwen3 MX configurations' that clearly indicates both the action and the affected configurations.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

📝 Coding Plan

• Generate coding plan for human review comments

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Tip

CodeRabbit can use TruffleHog to scan for secrets in your code with verification capabilities.

Add a TruffleHog config file (e.g. trufflehog-config.yml, trufflehog.yml) to your project to customize detectors and scanning behavior. The tool runs only when a config file is present.

[ko3n1g]

Do we need to update golden values?

[dingqingy-nv]

Yes, will update golden values