Repository navigation
Simplify partial messages - #671
Conversation
| @@ -1,290 +0,0 @@ | |||
| package partialmessages | |||
|
|
|||
There was a problem hiding this comment.
This was used when the Message interface had more methods and the library side expected more from the application. With these changes, the library side expects very little from the Message implementation, so these invariants are no longer required by the library. An application should have similar tests though, and they can be more specific.
0d7897b to
2613503
Compare
| type PeerState struct { | ||
| // Opaque (to gossipsub) state used by the application to track a peer's | ||
| // state (e.g. PartsMetadata) | ||
| RecvdState any |
There was a problem hiding this comment.
NIT:
| RecvdState any | |
| ReceivedState any |
I also like RecvState the 4 letters in Recv line up nicely with Sent
| // will not see the latest PeerState. | ||
| // | ||
| // If the rpc should be ignored, the application can return an empty PeerState. | ||
| OnIncomingRPC func(from peer.ID, peerState PeerState, rpc *pb.PartialMessagesExtension) (PeerState, error) |
There was a problem hiding this comment.
| OnIncomingRPC func(from peer.ID, peerState PeerState, rpc *pb.PartialMessagesExtension) (PeerState, error) | |
| OnIncomingRPC func(from peer.ID, previousPeerState PeerState, rpc *pb.PartialMessagesExtension) (newPeerState PeerState, _ error) |
| // SHOULD still return relevant partsMetadataToSend. | ||
| // | ||
| // The implementation should update: | ||
| // - peerState.RecvdState to remember what parts we've sent to this peer. |
There was a problem hiding this comment.
I think this should be
| // - peerState.RecvdState to remember what parts we've sent to this peer. | |
| // - peerState.RecvdState to remember what parts the peer has sent to us. |
There was a problem hiding this comment.
hm, this can probably be rephrased. The original is closer to what I want because we are updating their recvdState to remember that we sent them part X. That way, next time we are publishing, we don't resend them the part we've previously sent.
There was a problem hiding this comment.
I'm confused. This is the peer's partsmetadata that we've received from this peer, right?
The very next line:
// peerState.RecvdState is nil if the node has not seen its peer's
// partsMetadata, applications can still eagerly push data in this case.
| return err | ||
| } | ||
| if !pState.IsZero() { | ||
| state, err := e.groupState(topic, groupID, true, from) |
There was a problem hiding this comment.
NIT: I think this will read better if we rename state -> gState
| pState = &peerState{} | ||
| state.peerState[p] = pState | ||
| // Ask the application for message data | ||
| pState, pm, partsMetadata, err := partial.ForPeer(p, requestedPartial, pState) |
There was a problem hiding this comment.
There is one problem here. Either we solve it here or document it well.
Basically, we pass a copy of the peerState to the "ForPeer", "GossipForPeer" and "OnIncomingRPC" functions. Then, if these functions do not return an error, we update the peerState with what these functions return.
The problem is that if "RecvdState" and "SentState" are reference types(
for eg: a pointer), the app layer can shoot itself in the foot by updating them in-place and then return an error anyways. So either we do a deep copy of the PeerState here before passing it to the app layer and restore the state with the deep copy if there's an error or the app does it.
There was a problem hiding this comment.
- We can mandate that it is the app layer's responsibility to not mutate the PeerState in place if it ultimately errors out (by operating on a deep copy of the passed PeerState and then returning the modified deep copy here) but we should document it well here
OR
- Add a deep copy interface that the app layer must implement for PeerState -> use that to create a deep copy here before passing the PeerState to the app callbacks and then update the mapping with the returned PeerState only if there is no error (which we already do).
There was a problem hiding this comment.
For now, I've handled this in prysm by doing a deep copy of the PeerState before mutating it.
See OffchainLabs/prysm@eda07c0.
There was a problem hiding this comment.
So the latest version here makes it very clear that the app is responsible for state updates. Does that work?
The original motivations for this are gone with the future partialmessages.Message interface being simpler
PeerState is now only used by the application, but its lifetime is managed by gossipsub. This attempts to strike a reasonable tradeoff between giving the application complete responsibility of handling every peer's state, and the library from having to know too much about the structure of PartsMetadata. Applications can use the PeerState to store objects representing the node's view of a remote's state, and the node's view of the remote's view of our state. Applications SHOULD use this to avoid sending redundant or duplicate data.
We only need a single function from the application. Have the application provide this function in PublishPartial instead of a interface type.
It was only a wrapper around the group state map, and topicPeer state map. The latter is already capture by gossipsub, so we don't want to duplicate state.
2f962f7 to
d6e34cf
Compare
2613503 to
63d7f56
Compare
PeerState is now only used by the application, but its lifetime is
managed by gossipsub. This attempts to strike a reasonable tradeoff
between giving the application complete responsibility of handling every
peer's state, and the library from having to know too much about the
structure of PartsMetadata.
Applications can use the PeerState to store objects representing the
node's view of a remote's state, and the node's view of the remote's
view of our state. Applications SHOULD use this to avoid sending
redundant or duplicate data.
A future PR may use generics here instead of
any, but otherwise very similar.