Repository navigation
[misc] feat: trace rollout generation and tool calls using weave - #2345
Merged
Merged
Conversation
chenhaiq
requested review from
PeterSH6,
eric-haibin-lin,
tongyx361,
vermouth1992 and
zhaochenyang20
as code owners
July 3, 2025 09:29
chenhaiq
requested review from
SwordFaith and
wuxibin89
and removed request for
PeterSH6,
eric-haibin-lin,
tongyx361,
vermouth1992 and
zhaochenyang20
July 3, 2025 09:29
Collaborator
|
@chenhaiq Haiquan, could you help to check wether this is also supported in SGLang? I am checking with our team also. Thanks! |
Collaborator
Author
sure，I will support sglang |
Collaborator
Author
It supports sglang async mode, please have a code review. |
wuxibin89
reviewed
Jul 8, 2025
| rollout_n += 1 | ||
| else: | ||
| rollout_n = 0 | ||
| trajectory_info.append({"step": step, "sample_index": index[i], "rollout_n": rollout_n}) |
Collaborator
There was a problem hiding this comment.
what is the meaning of rollout_n?
Collaborator
Author
There was a problem hiding this comment.
rollout_n is used to associate trace entry with actor_rollout_ref.rollout.n.
For example, when actor_rollout_ref.rollout.n=2, the _run_agent_loop method will be executed 2 times.
Both of the 2 _run_agent_loop calls have their own trace entry. Setting a rollout_n is used to tell which time it is traced from.
wuxibin89
reviewed
Jul 8, 2025
| yield | ||
|
|
||
|
|
||
| def rollout_trace_op(func): |
Collaborator
There was a problem hiding this comment.
Seems like this function can trace every function, include tooling, agent loop run, ...?
Collaborator
Author
There was a problem hiding this comment.
It can trace any method in an class instance.
wuxibin89
approved these changes
Jul 8, 2025
| .. code:: bash | ||
|
|
||
| +trainer.rollout_trace.token2text=True # default to False for better performance | ||
|
|
Collaborator
There was a problem hiding this comment.
@chenhaiq could u add the visualization result from this PR to the documnetation page as well? please upload the image to https://github.com/eric-haibin-lin/verl-community/tree/main/docs
Collaborator
Author
|
corresponding feature in mlflow: 
|
lkc233
pushed a commit
to lkc233/verl
that referenced
this pull request
Jul 10, 2025
…l-project#2345) ### What does this PR do? Provide rollout generation and tool calls details in wandb weave to help debugging agentic RL. 2 new interfaces: 1. rollout_trace_attr contextmanager: used to mark sample_index、step、rollout_n and experience name for a trajectory. 2. rollout_trace_op decorator：mark the method to trace. It must be a method of an instance. related issue verl-project#2188 ### Checklist Before Starting - [X] Search for similar PRs. Paste at least one query link here: ... - [X] Format the PR title as `[{modules}] {type}: {description}` (This will be checked by the CI) - `{modules}` include `fsdp`, `megatron`, `sglang`, `vllm`, `rollout`, `trainer`, `ci`, `training_utils`, `recipe`, `hardware`, `deployment`, `ray`, `worker`, `single_controller`, `misc`, `perf`, `model`, `algo`, `env`, `tool`, `ckpt`, `doc`, `data` - If this PR involves multiple modules, separate them with `,` like `[megatron, fsdp, doc]` - `{type}` is in `feat`, `fix`, `refactor`, `chore`, `test` - If this PR breaks any API (CLI arguments, config, function signature, etc.), add `[BREAKING]` to the beginning of the title. - Example: `[BREAKING][fsdp, megatron] feat: dynamic batching` ### Test <img width="1910" alt="截屏2025-07-03 下午4 09 58" src="https://github.com/user-attachments/assets/ff30bbca-f9c8-434f-a3c2-0e333d16fa68" /> <img width="1895" alt="截屏2025-07-03 下午4 11 27" src="https://github.com/user-attachments/assets/0b9ed8db-58a7-4769-88fb-bda204dc9fc8" /> ### API and Usage Example options: +trainer.rollout_trace.backend=weave: only wandb weave is support in this PR. Leave the reset of trace tool to the community. +trainer.rollout_trace.token2text=False: whether append decoded text in result of run method. ### High-Level Design n/a ### Specific Changes Only works for async rollout from agent loop. No effect for sync rollout. ### Checklist Before Submitting > [!IMPORTANT] > Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review. - [x] Read the [Contribute Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide). - [x] Apply [pre-commit checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting): `pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always` - [x] Add / Update [the documentation](https://github.com/volcengine/verl/tree/main/docs). - [x] Add unit or end-to-end test(s) to [the CI workflow](https://github.com/volcengine/verl/tree/main/.github/workflows) to cover all the code. If not feasible, explain why: ... - [x] Once your PR is ready for CI, send a message in [the `ci-request` channel](https://verl-project.slack.com/archives/C091TCESWB1) in [the `verl` Slack workspace](https://join.slack.com/t/verl-project/shared_invite/zt-3855yhg8g-CTkqXu~hKojPCmo7k_yXTQ).
ArronHZG
pushed a commit
to imh966/verl
that referenced
this pull request
Jul 10, 2025
…l-project#2345) ### What does this PR do? Provide rollout generation and tool calls details in wandb weave to help debugging agentic RL. 2 new interfaces: 1. rollout_trace_attr contextmanager: used to mark sample_index、step、rollout_n and experience name for a trajectory. 2. rollout_trace_op decorator：mark the method to trace. It must be a method of an instance. related issue verl-project#2188 ### Checklist Before Starting - [X] Search for similar PRs. Paste at least one query link here: ... - [X] Format the PR title as `[{modules}] {type}: {description}` (This will be checked by the CI) - `{modules}` include `fsdp`, `megatron`, `sglang`, `vllm`, `rollout`, `trainer`, `ci`, `training_utils`, `recipe`, `hardware`, `deployment`, `ray`, `worker`, `single_controller`, `misc`, `perf`, `model`, `algo`, `env`, `tool`, `ckpt`, `doc`, `data` - If this PR involves multiple modules, separate them with `,` like `[megatron, fsdp, doc]` - `{type}` is in `feat`, `fix`, `refactor`, `chore`, `test` - If this PR breaks any API (CLI arguments, config, function signature, etc.), add `[BREAKING]` to the beginning of the title. - Example: `[BREAKING][fsdp, megatron] feat: dynamic batching` ### Test <img width="1910" alt="截屏2025-07-03 下午4 09 58" src="https://github.com/user-attachments/assets/ff30bbca-f9c8-434f-a3c2-0e333d16fa68" /> <img width="1895" alt="截屏2025-07-03 下午4 11 27" src="https://github.com/user-attachments/assets/0b9ed8db-58a7-4769-88fb-bda204dc9fc8" /> ### API and Usage Example options: +trainer.rollout_trace.backend=weave: only wandb weave is support in this PR. Leave the reset of trace tool to the community. +trainer.rollout_trace.token2text=False: whether append decoded text in result of run method. ### High-Level Design n/a ### Specific Changes Only works for async rollout from agent loop. No effect for sync rollout. ### Checklist Before Submitting > [!IMPORTANT] > Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review. - [x] Read the [Contribute Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide). - [x] Apply [pre-commit checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting): `pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always` - [x] Add / Update [the documentation](https://github.com/volcengine/verl/tree/main/docs). - [x] Add unit or end-to-end test(s) to [the CI workflow](https://github.com/volcengine/verl/tree/main/.github/workflows) to cover all the code. If not feasible, explain why: ... - [x] Once your PR is ready for CI, send a message in [the `ci-request` channel](https://verl-project.slack.com/archives/C091TCESWB1) in [the `verl` Slack workspace](https://join.slack.com/t/verl-project/shared_invite/zt-3855yhg8g-CTkqXu~hKojPCmo7k_yXTQ).
7 tasks done
2 of 7 tasks
oseyosey
pushed a commit
to oseyosey/verl
that referenced
this pull request
Jul 28, 2025
…l-project#2345) ### What does this PR do? Provide rollout generation and tool calls details in wandb weave to help debugging agentic RL. 2 new interfaces: 1. rollout_trace_attr contextmanager: used to mark sample_index、step、rollout_n and experience name for a trajectory. 2. rollout_trace_op decorator：mark the method to trace. It must be a method of an instance. related issue verl-project#2188 ### Checklist Before Starting - [X] Search for similar PRs. Paste at least one query link here: ... - [X] Format the PR title as `[{modules}] {type}: {description}` (This will be checked by the CI) - `{modules}` include `fsdp`, `megatron`, `sglang`, `vllm`, `rollout`, `trainer`, `ci`, `training_utils`, `recipe`, `hardware`, `deployment`, `ray`, `worker`, `single_controller`, `misc`, `perf`, `model`, `algo`, `env`, `tool`, `ckpt`, `doc`, `data` - If this PR involves multiple modules, separate them with `,` like `[megatron, fsdp, doc]` - `{type}` is in `feat`, `fix`, `refactor`, `chore`, `test` - If this PR breaks any API (CLI arguments, config, function signature, etc.), add `[BREAKING]` to the beginning of the title. - Example: `[BREAKING][fsdp, megatron] feat: dynamic batching` ### Test <img width="1910" alt="截屏2025-07-03 下午4 09 58" src="https://github.com/user-attachments/assets/ff30bbca-f9c8-434f-a3c2-0e333d16fa68" /> <img width="1895" alt="截屏2025-07-03 下午4 11 27" src="https://github.com/user-attachments/assets/0b9ed8db-58a7-4769-88fb-bda204dc9fc8" /> ### API and Usage Example options: +trainer.rollout_trace.backend=weave: only wandb weave is support in this PR. Leave the reset of trace tool to the community. +trainer.rollout_trace.token2text=False: whether append decoded text in result of run method. ### High-Level Design n/a ### Specific Changes Only works for async rollout from agent loop. No effect for sync rollout. ### Checklist Before Submitting > [!IMPORTANT] > Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review. - [x] Read the [Contribute Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide). - [x] Apply [pre-commit checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting): `pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always` - [x] Add / Update [the documentation](https://github.com/volcengine/verl/tree/main/docs). - [x] Add unit or end-to-end test(s) to [the CI workflow](https://github.com/volcengine/verl/tree/main/.github/workflows) to cover all the code. If not feasible, explain why: ... - [x] Once your PR is ready for CI, send a message in [the `ci-request` channel](https://verl-project.slack.com/archives/C091TCESWB1) in [the `verl` Slack workspace](https://join.slack.com/t/verl-project/shared_invite/zt-3855yhg8g-CTkqXu~hKojPCmo7k_yXTQ).
Juniper1021
pushed a commit
to Juniper1021/verl
that referenced
this pull request
Aug 7, 2025
…l-project#2345) ### What does this PR do? Provide rollout generation and tool calls details in wandb weave to help debugging agentic RL. 2 new interfaces: 1. rollout_trace_attr contextmanager: used to mark sample_index、step、rollout_n and experience name for a trajectory. 2. rollout_trace_op decorator：mark the method to trace. It must be a method of an instance. related issue verl-project#2188 ### Checklist Before Starting - [X] Search for similar PRs. Paste at least one query link here: ... - [X] Format the PR title as `[{modules}] {type}: {description}` (This will be checked by the CI) - `{modules}` include `fsdp`, `megatron`, `sglang`, `vllm`, `rollout`, `trainer`, `ci`, `training_utils`, `recipe`, `hardware`, `deployment`, `ray`, `worker`, `single_controller`, `misc`, `perf`, `model`, `algo`, `env`, `tool`, `ckpt`, `doc`, `data` - If this PR involves multiple modules, separate them with `,` like `[megatron, fsdp, doc]` - `{type}` is in `feat`, `fix`, `refactor`, `chore`, `test` - If this PR breaks any API (CLI arguments, config, function signature, etc.), add `[BREAKING]` to the beginning of the title. - Example: `[BREAKING][fsdp, megatron] feat: dynamic batching` ### Test <img width="1910" alt="截屏2025-07-03 下午4 09 58" src="https://github.com/user-attachments/assets/ff30bbca-f9c8-434f-a3c2-0e333d16fa68" /> <img width="1895" alt="截屏2025-07-03 下午4 11 27" src="https://github.com/user-attachments/assets/0b9ed8db-58a7-4769-88fb-bda204dc9fc8" /> ### API and Usage Example options: +trainer.rollout_trace.backend=weave: only wandb weave is support in this PR. Leave the reset of trace tool to the community. +trainer.rollout_trace.token2text=False: whether append decoded text in result of run method. ### High-Level Design n/a ### Specific Changes Only works for async rollout from agent loop. No effect for sync rollout. ### Checklist Before Submitting > [!IMPORTANT] > Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review. - [x] Read the [Contribute Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide). - [x] Apply [pre-commit checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting): `pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always` - [x] Add / Update [the documentation](https://github.com/volcengine/verl/tree/main/docs). - [x] Add unit or end-to-end test(s) to [the CI workflow](https://github.com/volcengine/verl/tree/main/.github/workflows) to cover all the code. If not feasible, explain why: ... - [x] Once your PR is ready for CI, send a message in [the `ci-request` channel](https://verl-project.slack.com/archives/C091TCESWB1) in [the `verl` Slack workspace](https://join.slack.com/t/verl-project/shared_invite/zt-3855yhg8g-CTkqXu~hKojPCmo7k_yXTQ).
whatadayG
pushed a commit
to whatadayG/verl
that referenced
this pull request
Sep 5, 2025
…l-project#2345) ### What does this PR do? Provide rollout generation and tool calls details in wandb weave to help debugging agentic RL. 2 new interfaces: 1. rollout_trace_attr contextmanager: used to mark sample_index、step、rollout_n and experience name for a trajectory. 2. rollout_trace_op decorator：mark the method to trace. It must be a method of an instance. related issue verl-project#2188 ### Checklist Before Starting - [X] Search for similar PRs. Paste at least one query link here: ... - [X] Format the PR title as `[{modules}] {type}: {description}` (This will be checked by the CI) - `{modules}` include `fsdp`, `megatron`, `sglang`, `vllm`, `rollout`, `trainer`, `ci`, `training_utils`, `recipe`, `hardware`, `deployment`, `ray`, `worker`, `single_controller`, `misc`, `perf`, `model`, `algo`, `env`, `tool`, `ckpt`, `doc`, `data` - If this PR involves multiple modules, separate them with `,` like `[megatron, fsdp, doc]` - `{type}` is in `feat`, `fix`, `refactor`, `chore`, `test` - If this PR breaks any API (CLI arguments, config, function signature, etc.), add `[BREAKING]` to the beginning of the title. - Example: `[BREAKING][fsdp, megatron] feat: dynamic batching` ### Test <img width="1910" alt="截屏2025-07-03 下午4 09 58" src="https://github.com/user-attachments/assets/ff30bbca-f9c8-434f-a3c2-0e333d16fa68" /> <img width="1895" alt="截屏2025-07-03 下午4 11 27" src="https://github.com/user-attachments/assets/0b9ed8db-58a7-4769-88fb-bda204dc9fc8" /> ### API and Usage Example options: +trainer.rollout_trace.backend=weave: only wandb weave is support in this PR. Leave the reset of trace tool to the community. +trainer.rollout_trace.token2text=False: whether append decoded text in result of run method. ### High-Level Design n/a ### Specific Changes Only works for async rollout from agent loop. No effect for sync rollout. ### Checklist Before Submitting > [!IMPORTANT] > Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review. - [x] Read the [Contribute Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide). - [x] Apply [pre-commit checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting): `pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always` - [x] Add / Update [the documentation](https://github.com/volcengine/verl/tree/main/docs). - [x] Add unit or end-to-end test(s) to [the CI workflow](https://github.com/volcengine/verl/tree/main/.github/workflows) to cover all the code. If not feasible, explain why: ... - [x] Once your PR is ready for CI, send a message in [the `ci-request` channel](https://verl-project.slack.com/archives/C091TCESWB1) in [the `verl` Slack workspace](https://join.slack.com/t/verl-project/shared_invite/zt-3855yhg8g-CTkqXu~hKojPCmo7k_yXTQ).
chenjiaoAngel
added a commit
to chenjiaoAngel/verl
that referenced
this pull request
Nov 14, 2025
…l-project#2345) ### What does this PR do? Provide rollout generation and tool calls details in wandb weave to help debugging agentic RL. 2 new interfaces: 1. rollout_trace_attr contextmanager: used to mark sample_index、step、rollout_n and experience name for a trajectory. 2. rollout_trace_op decorator：mark the method to trace. It must be a method of an instance. related issue verl-project#2188 ### Checklist Before Starting - [X] Search for similar PRs. Paste at least one query link here: ... - [X] Format the PR title as `[{modules}] {type}: {description}` (This will be checked by the CI) - `{modules}` include `fsdp`, `megatron`, `sglang`, `vllm`, `rollout`, `trainer`, `ci`, `training_utils`, `recipe`, `hardware`, `deployment`, `ray`, `worker`, `single_controller`, `misc`, `perf`, `model`, `algo`, `env`, `tool`, `ckpt`, `doc`, `data` - If this PR involves multiple modules, separate them with `,` like `[megatron, fsdp, doc]` - `{type}` is in `feat`, `fix`, `refactor`, `chore`, `test` - If this PR breaks any API (CLI arguments, config, function signature, etc.), add `[BREAKING]` to the beginning of the title. - Example: `[BREAKING][fsdp, megatron] feat: dynamic batching` ### Test <img width="1910" alt="截屏2025-07-03 下午4 09 58" src="https://github.com/user-attachments/assets/ff30bbca-f9c8-434f-a3c2-0e333d16fa68" /> <img width="1895" alt="截屏2025-07-03 下午4 11 27" src="https://github.com/user-attachments/assets/0b9ed8db-58a7-4769-88fb-bda204dc9fc8" /> ### API and Usage Example options: +trainer.rollout_trace.backend=weave: only wandb weave is support in this PR. Leave the reset of trace tool to the community. +trainer.rollout_trace.token2text=False: whether append decoded text in result of run method. ### High-Level Design n/a ### Specific Changes Only works for async rollout from agent loop. No effect for sync rollout. ### Checklist Before Submitting > [!IMPORTANT] > Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review. - [x] Read the [Contribute Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide). - [x] Apply [pre-commit checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting): `pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always` - [x] Add / Update [the documentation](https://github.com/volcengine/verl/tree/main/docs). - [x] Add unit or end-to-end test(s) to [the CI workflow](https://github.com/volcengine/verl/tree/main/.github/workflows) to cover all the code. If not feasible, explain why: ... - [x] Once your PR is ready for CI, send a message in [the `ci-request` channel](https://verl-project.slack.com/archives/C091TCESWB1) in [the `verl` Slack workspace](https://join.slack.com/t/verl-project/shared_invite/zt-3855yhg8g-CTkqXu~hKojPCmo7k_yXTQ).
TimurTaepov
pushed a commit
to giorgossideris/verl
that referenced
this pull request
Dec 20, 2025
…l-project#2345) ### What does this PR do? Provide rollout generation and tool calls details in wandb weave to help debugging agentic RL. 2 new interfaces: 1. rollout_trace_attr contextmanager: used to mark sample_index、step、rollout_n and experience name for a trajectory. 2. rollout_trace_op decorator：mark the method to trace. It must be a method of an instance. related issue verl-project#2188 ### Checklist Before Starting - [X] Search for similar PRs. Paste at least one query link here: ... - [X] Format the PR title as `[{modules}] {type}: {description}` (This will be checked by the CI) - `{modules}` include `fsdp`, `megatron`, `sglang`, `vllm`, `rollout`, `trainer`, `ci`, `training_utils`, `recipe`, `hardware`, `deployment`, `ray`, `worker`, `single_controller`, `misc`, `perf`, `model`, `algo`, `env`, `tool`, `ckpt`, `doc`, `data` - If this PR involves multiple modules, separate them with `,` like `[megatron, fsdp, doc]` - `{type}` is in `feat`, `fix`, `refactor`, `chore`, `test` - If this PR breaks any API (CLI arguments, config, function signature, etc.), add `[BREAKING]` to the beginning of the title. - Example: `[BREAKING][fsdp, megatron] feat: dynamic batching` ### Test <img width="1910" alt="截屏2025-07-03 下午4 09 58" src="https://github.com/user-attachments/assets/ff30bbca-f9c8-434f-a3c2-0e333d16fa68" /> <img width="1895" alt="截屏2025-07-03 下午4 11 27" src="https://github.com/user-attachments/assets/0b9ed8db-58a7-4769-88fb-bda204dc9fc8" /> ### API and Usage Example options: +trainer.rollout_trace.backend=weave: only wandb weave is support in this PR. Leave the reset of trace tool to the community. +trainer.rollout_trace.token2text=False: whether append decoded text in result of run method. ### High-Level Design n/a ### Specific Changes Only works for async rollout from agent loop. No effect for sync rollout. ### Checklist Before Submitting > [!IMPORTANT] > Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review. - [x] Read the [Contribute Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide). - [x] Apply [pre-commit checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting): `pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always` - [x] Add / Update [the documentation](https://github.com/volcengine/verl/tree/main/docs). - [x] Add unit or end-to-end test(s) to [the CI workflow](https://github.com/volcengine/verl/tree/main/.github/workflows) to cover all the code. If not feasible, explain why: ... - [x] Once your PR is ready for CI, send a message in [the `ci-request` channel](https://verl-project.slack.com/archives/C091TCESWB1) in [the `verl` Slack workspace](https://join.slack.com/t/verl-project/shared_invite/zt-3855yhg8g-CTkqXu~hKojPCmo7k_yXTQ).
oseyosey
pushed a commit
to oseyosey/verl
that referenced
this pull request
Jan 20, 2026
…l-project#2345) ### What does this PR do? Provide rollout generation and tool calls details in wandb weave to help debugging agentic RL. 2 new interfaces: 1. rollout_trace_attr contextmanager: used to mark sample_index、step、rollout_n and experience name for a trajectory. 2. rollout_trace_op decorator：mark the method to trace. It must be a method of an instance. related issue verl-project#2188 ### Checklist Before Starting - [X] Search for similar PRs. Paste at least one query link here: ... - [X] Format the PR title as `[{modules}] {type}: {description}` (This will be checked by the CI) - `{modules}` include `fsdp`, `megatron`, `sglang`, `vllm`, `rollout`, `trainer`, `ci`, `training_utils`, `recipe`, `hardware`, `deployment`, `ray`, `worker`, `single_controller`, `misc`, `perf`, `model`, `algo`, `env`, `tool`, `ckpt`, `doc`, `data` - If this PR involves multiple modules, separate them with `,` like `[megatron, fsdp, doc]` - `{type}` is in `feat`, `fix`, `refactor`, `chore`, `test` - If this PR breaks any API (CLI arguments, config, function signature, etc.), add `[BREAKING]` to the beginning of the title. - Example: `[BREAKING][fsdp, megatron] feat: dynamic batching` ### Test <img width="1910" alt="截屏2025-07-03 下午4 09 58" src="https://github.com/user-attachments/assets/ff30bbca-f9c8-434f-a3c2-0e333d16fa68" /> <img width="1895" alt="截屏2025-07-03 下午4 11 27" src="https://github.com/user-attachments/assets/0b9ed8db-58a7-4769-88fb-bda204dc9fc8" /> ### API and Usage Example options: +trainer.rollout_trace.backend=weave: only wandb weave is support in this PR. Leave the reset of trace tool to the community. +trainer.rollout_trace.token2text=False: whether append decoded text in result of run method. ### High-Level Design n/a ### Specific Changes Only works for async rollout from agent loop. No effect for sync rollout. ### Checklist Before Submitting > [!IMPORTANT] > Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review. - [x] Read the [Contribute Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide). - [x] Apply [pre-commit checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting): `pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always` - [x] Add / Update [the documentation](https://github.com/volcengine/verl/tree/main/docs). - [x] Add unit or end-to-end test(s) to [the CI workflow](https://github.com/volcengine/verl/tree/main/.github/workflows) to cover all the code. If not feasible, explain why: ... - [x] Once your PR is ready for CI, send a message in [the `ci-request` channel](https://verl-project.slack.com/archives/C091TCESWB1) in [the `verl` Slack workspace](https://join.slack.com/t/verl-project/shared_invite/zt-3855yhg8g-CTkqXu~hKojPCmo7k_yXTQ).
vyomakesh0728
added a commit
to vyomakesh0728/verl
that referenced
this pull request
Jan 22, 2026
…l-project#2345) ### What does this PR do? Provide rollout generation and tool calls details in wandb weave to help debugging agentic RL. 2 new interfaces: 1. rollout_trace_attr contextmanager: used to mark sample_index、step、rollout_n and experience name for a trajectory. 2. rollout_trace_op decorator：mark the method to trace. It must be a method of an instance. related issue verl-project#2188 ### Checklist Before Starting - [X] Search for similar PRs. Paste at least one query link here: ... - [X] Format the PR title as `[{modules}] {type}: {description}` (This will be checked by the CI) - `{modules}` include `fsdp`, `megatron`, `sglang`, `vllm`, `rollout`, `trainer`, `ci`, `training_utils`, `recipe`, `hardware`, `deployment`, `ray`, `worker`, `single_controller`, `misc`, `perf`, `model`, `algo`, `env`, `tool`, `ckpt`, `doc`, `data` - If this PR involves multiple modules, separate them with `,` like `[megatron, fsdp, doc]` - `{type}` is in `feat`, `fix`, `refactor`, `chore`, `test` - If this PR breaks any API (CLI arguments, config, function signature, etc.), add `[BREAKING]` to the beginning of the title. - Example: `[BREAKING][fsdp, megatron] feat: dynamic batching` ### Test <img width="1910" alt="截屏2025-07-03 下午4 09 58" src="https://github.com/user-attachments/assets/ff30bbca-f9c8-434f-a3c2-0e333d16fa68" /> <img width="1895" alt="截屏2025-07-03 下午4 11 27" src="https://github.com/user-attachments/assets/0b9ed8db-58a7-4769-88fb-bda204dc9fc8" /> ### API and Usage Example options: +trainer.rollout_trace.backend=weave: only wandb weave is support in this PR. Leave the reset of trace tool to the community. +trainer.rollout_trace.token2text=False: whether append decoded text in result of run method. ### High-Level Design n/a ### Specific Changes Only works for async rollout from agent loop. No effect for sync rollout. ### Checklist Before Submitting > [!IMPORTANT] > Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review. - [x] Read the [Contribute Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide). - [x] Apply [pre-commit checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting): `pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always` - [x] Add / Update [the documentation](https://github.com/volcengine/verl/tree/main/docs). - [x] Add unit or end-to-end test(s) to [the CI workflow](https://github.com/volcengine/verl/tree/main/.github/workflows) to cover all the code. If not feasible, explain why: ... - [x] Once your PR is ready for CI, send a message in [the `ci-request` channel](https://verl-project.slack.com/archives/C091TCESWB1) in [the `verl` Slack workspace](https://join.slack.com/t/verl-project/shared_invite/zt-3855yhg8g-CTkqXu~hKojPCmo7k_yXTQ).
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Provide rollout generation and tool calls details in wandb weave to help debugging agentic RL.
2 new interfaces:
related issue #2188
Checklist Before Starting
[{modules}] {type}: {description}(This will be checked by the CI){modules}includefsdp,megatron,sglang,vllm,rollout,trainer,ci,training_utils,recipe,hardware,deployment,ray,worker,single_controller,misc,perf,model,algo,env,tool,ckpt,doc,data,like[megatron, fsdp, doc]{type}is infeat,fix,refactor,chore,test[BREAKING]to the beginning of the title.[BREAKING][fsdp, megatron] feat: dynamic batchingTest
API and Usage Example
options:
+trainer.rollout_trace.backend=weave: only wandb weave is support in this PR. Leave the reset of trace tool to the community.
+trainer.rollout_trace.token2text=False: whether append decoded text in result of run method.
High-Level Design
n/a
Specific Changes
Only works for async rollout from agent loop. No effect for sync rollout.
Checklist Before Submitting
Important
Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.
pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=alwaysci-requestchannel in theverlSlack workspace.