Skip to content

Simplify stream_output - #2398

Merged
merrymercy merged 19 commits into
mainfrom
pr-fix-streaming
Dec 8, 2024
Merged

Simplify stream_output#2398
merrymercy merged 19 commits into
mainfrom
pr-fix-streaming

Conversation

@merrymercy

@merrymercy merrymercy commented Dec 8, 2024 •
edited
Loading

Copy link
Copy Markdown
Collaborator
  • Flatten the tuple for logprobs. It makes zmq IPC faster. This is useful when using logprob + streaming.

@merrymercy
merrymercy merged commit a6ca736 into main Dec 8, 2024
@merrymercy
merrymercy deleted the pr-fix-streaming branch December 8, 2024 20:27
timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025
@merrymercy
Simplify stream_output (sgl-project#2398)
29ee0a1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Ying1123 Ying1123 Awaiting requested review from Ying1123 Ying1123 is a code owner
@hnyls2002 hnyls2002 Awaiting requested review from hnyls2002 hnyls2002 is a code owner
@zhyncs zhyncs Awaiting requested review from zhyncs
@ispobock ispobock Awaiting requested review from ispobock ispobock is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@merrymercy