Skip to content

New net interface - #69

Merged
jbenet merged 221 commits into
masterfrom
net
Sep 22, 2014
Merged

jbenet merged 221 commits into
masterfrom
net

Conversation

@jbenet

@jbenet jbenet commented Sep 14, 2014

Copy link
Copy Markdown
Member

No description provided.

@jbenet jbenet added the status/in-progress In progress label Sep 14, 2014
@jbenet

jbenet commented Sep 14, 2014

Copy link
Copy Markdown
Member Author

warning: not yet wired.

@btc btc mentioned this pull request Sep 14, 2014
@jbenet

jbenet commented Sep 14, 2014

Copy link
Copy Markdown
Member Author

#75 #74

Comment thread net/service/request.go

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@perfmode @whyrusleeping heads up -- Request uses the per-msg RequestID nonce to also identify whether it's a request or a response (based on the first bit). (lmk if you think this is too sketchy)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When can r == nil?

My concern would be around future-proofing. Will this representation be resistant to errors introduced by updates and versioning incompatibilities?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ehhh... im going to lean towards 'sketchy'. Like, yeah it works, but it should be commented to hell and back if were going to keep it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@perfmode r == nil when messages do not want a RequestID associated -- i.e. they're not requests.

@whyrusleeping Yeah, it's sketchy. It feels so wrong to use a whole protobuf for a bit thought >.< -- i guess it is technically TRTTD.

Comment thread routing/dht/dht.go

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we have some sort of extra functionality added to multiaddr? Maybe add pluggable protocols so ipfs can specify the 'ipfs peer' protocol which denotes an id that is expeced from the connection?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, sounds good. we can allocate a multiaddr number and all.

think b58 is good enough to have multiaddr default to for protocols whose string representations it doesn't know? could do hex. Oh damnit b58 will run into trouble on case sensitive OSes....

Comment thread routing/dht/query.go

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hrm... having a reference to outself here might be useful. I had issues without it before.

@btc

btc commented Sep 19, 2014

Copy link
Copy Markdown
Contributor

@jbenet Recall services register handlers on the network. These handlers are called when inbound messages arrive.

 HandleMessage(context.Context, msg.NetMessage) (msg.NetMessage, error)

What does it mean for the handler to return an error to the network layer?

@jbenet

jbenet commented Sep 19, 2014

Copy link
Copy Markdown
Member Author

@perfmode good point, that doesn't make much sense. See https://github.com/jbenet/go-ipfs/blob/net/net/service/service.go#L183-L188 Think it should just return a message, and handle/log/whatever its own error?

@btc

btc commented Sep 20, 2014

Copy link
Copy Markdown
Contributor

@jbenet @whyrusleeping merged this into net already, but still want to make sure you're aware.

#91

Brian Tiger Chow added 14 commits September 22, 2014 04:06
changing the bitswap interace breaks tests and makes things a bit
difficult going forward. I think I have a temporary solution to replace
the async method.

this commit partially reverts changes from:

ec50703

ec50703
until construction is complete on the actual async method

reverts changes from ec50703

ec50703
fix(bitswap) pass go vet

fixes #97

#97
true -> always send to peer

false -> use ledger-based strategy described in IPFS paper draft 3
failing due to nil pointer dereference
@btc

btc commented Sep 22, 2014

Copy link
Copy Markdown
Contributor

@jbenet @whyrusleeping

:shipit:

jbenet added a commit that referenced this pull request Sep 22, 2014
@jbenet
jbenet merged commit 676d9c5 into master Sep 22, 2014
@jbenet jbenet removed the status/in-progress In progress label Sep 22, 2014
@jbenet

jbenet commented Sep 22, 2014

Copy link
Copy Markdown
Member Author

@btc

btc commented Sep 22, 2014

Copy link
Copy Markdown
Contributor

AHHHHHHHHHHHHHHH!!!!

@btc

btc commented Sep 22, 2014

Copy link
Copy Markdown
Contributor

The smile on my face right now.

@jbenet

jbenet commented Sep 22, 2014

Copy link
Copy Markdown
Member Author
Warning: remote port forwarding failed for listen port 52698
From github.com:jbenet/go-ipfs
 * branch            master     -> FETCH_HEAD
 Dockerfile                                                                 |  19 ++++
 Godeps/Godeps.json                                                         |  15 ++-
 Godeps/_workspace/src/code.google.com/p/go.net/context/context.go          | 431 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 Godeps/_workspace/src/code.google.com/p/go.net/context/context_test.go     | 553 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 Godeps/_workspace/src/code.google.com/p/go.net/context/withtimeout_test.go |  26 +++++
 Godeps/_workspace/src/github.com/jbenet/go-multiaddr/codec.go              |  28 +++--
 Godeps/_workspace/src/github.com/jbenet/go-multiaddr/index.go              |  21 +++-
 Godeps/_workspace/src/github.com/jbenet/go-multiaddr/multiaddr_test.go     |  38 ++++++-
 Godeps/_workspace/src/github.com/jbenet/go-multiaddr/net.go                |  77 +++++++++++++
 Godeps/_workspace/src/github.com/jbenet/go-multiaddr/net_test.go           |  49 +++++++++
 Godeps/_workspace/src/github.com/jbenet/go-multiaddr/protocols.go          |   5 +-
 Godeps/_workspace/src/github.com/tuxychandru/pubsub/README.md              |  30 ++++++
 Godeps/_workspace/src/github.com/tuxychandru/pubsub/pubsub.go              | 208 ++++++++++++++++++++++++++++++++++++
 Godeps/_workspace/src/github.com/tuxychandru/pubsub/pubsub_test.go         | 230 +++++++++++++++++++++++++++++++++++++++
 README.md                                                                  |  10 +-
 bitswap/bitswap.go                                                         | 282 ------------------------------------------------
 bitswap/ledger.go                                                          |  51 ---------
 blocks/blocks.go                                                           |   3 +-
 blocks/blocks_test.go                                                      |  25 +++++
 blockservice/blockservice.go                                               |  21 ++--
 blockstore/blockstore.go                                                   |  47 ++++++++
 blockstore/blockstore_test.go                                              |  55 ++++++++++
 cmd/ipfs/add.go                                                            | 100 +++--------------
 cmd/ipfs/cat.go                                                            |  29 ++---
 cmd/ipfs/init.go                                                           |  60 +++++++++--
 cmd/ipfs/ipfs.go                                                           |  28 ++++-
 cmd/ipfs/ls.go                                                             |  24 +++--
 cmd/ipfs/mount_unix.go                                                     |  28 ++++-
 cmd/ipfs/refs.go                                                           |  58 +++-------
 config/config.go                                                           |  18 ++--
 config/config_test.go                                                      |  17 ++-
 config/serialize.go                                                        |   5 +
 {cmd/ipfs => core/commands}/add.go                                         |  77 +++++--------
 core/commands/cat.go                                                       |  29 +++++
 core/commands/ls.go                                                        |  22 ++++
 core/commands/pin.go                                                       |  23 ++++
 core/commands/refs.go                                                      |  65 +++++++++++
 core/core.go                                                               | 142 +++++++++++++++++-------
 core/core_test.go                                                          |  28 ++++-
 core/datastore.go                                                          |   7 +-
 crypto/key_test.go                                                         |  54 ++++++++++
 crypto/spipe/Makefile                                                      |   8 ++
 identify/identify.go => crypto/spipe/handshake.go                          | 166 +++++++++++++++++-----------
 {identify => crypto/spipe}/message.pb.go                                   |  46 ++++----
 crypto/spipe/message.proto                                                 |  14 +++
 crypto/spipe/pipe.go                                                       |  76 +++++++++++++
 daemon/daemon.go                                                           | 102 ++++++++++++++++++
 daemon/daemon_client.go                                                    |  29 +++++
 daemon/daemon_test.go                                                      |  71 ++++++++++++
 dev.md                                                                     |   9 +-
 doc.go                                                                     |   2 +
 exchange/bitswap/bitswap.go                                                | 185 ++++++++++++++++++++++++++++++++
 exchange/bitswap/bitswap_test.go                                           | 301 +++++++++++++++++++++++++++++++++++++++++++++++++++
 exchange/bitswap/message/Makefile                                          |   8 ++
 exchange/bitswap/message/message.go                                        |  93 ++++++++++++++++
 {bitswap => exchange/bitswap/message}/message.pb.go                        |   2 +-
 {bitswap => exchange/bitswap/message}/message.proto                        |   2 +-
 exchange/bitswap/message/message_test.go                                   | 172 +++++++++++++++++++++++++++++
 exchange/bitswap/network/interface.go                                      |  53 +++++++++
 exchange/bitswap/network/net_message_adapter.go                            |  93 ++++++++++++++++
 exchange/bitswap/notifications/notifications.go                            |  55 ++++++++++
 exchange/bitswap/notifications/notifications_test.go                       |  58 ++++++++++
 exchange/bitswap/strategy/interface.go                                     |  32 ++++++
 exchange/bitswap/strategy/ledger.go                                        |  93 ++++++++++++++++
 exchange/bitswap/strategy/ledger_test.go                                   |  23 ++++
 bitswap/strategy.go => exchange/bitswap/strategy/math.go                   |  11 +-
 bitswap/strategy_test.go => exchange/bitswap/strategy/math_test.go         |   2 +-
 exchange/bitswap/strategy/strategy.go                                      | 114 ++++++++++++++++++++
 exchange/bitswap/strategy/strategy_test.go                                 | 103 ++++++++++++++++++
 exchange/bitswap/testnet/network.go                                        | 174 ++++++++++++++++++++++++++++++
 exchange/bitswap/testnet/network_test.go                                   | 138 ++++++++++++++++++++++++
 exchange/bitswap/testnet/routing.go                                        |  97 +++++++++++++++++
 exchange/bitswap/testnet/routing_test.go                                   | 157 +++++++++++++++++++++++++++
 exchange/interface.go                                                      |  20 ++++
 exchange/offline/offline.go                                                |  34 ++++++
 exchange/offline/offline_test.go                                           |  28 +++++
 identify/identify_test.go                                                  |  54 ----------
 identify/message.proto                                                     |  14 ---
 importer/importer.go                                                       |   7 +-
 importer/importer_test.go                                                  |  74 ++++++++++---
 importer/rabin.go                                                          | 134 +++++++++++++++++++++++
 importer/split_test.go                                                     |  48 ---------
 importer/splitting.go                                                      |  91 ++++------------
 merkledag/dagreader.go                                                     |  13 ++-
 merkledag/merkledag.go                                                     |   4 +-
 {swarm => net/conn}/conn.go                                                |  62 +++++++----
 {swarm => net/conn}/conn_test.go                                           |  25 ++---
 net/interface.go                                                           |  49 +++++++++
 net/message/message.go                                                     |  78 ++++++++++++++
 net/mux/Makefile                                                           |   8 ++
 net/mux/mux.go                                                             | 188 ++++++++++++++++++++++++++++++++
 net/mux/mux.pb.go                                                          |  90 ++++++++++++++++
 net/mux/mux.proto                                                          |  13 +++
 net/mux/mux_test.go                                                        | 276 +++++++++++++++++++++++++++++++++++++++++++++++
 net/net.go                                                                 | 109 +++++++++++++++++++
 net/net_test.go                                                            |   1 +
 net/service/Makefile                                                       |   8 ++
 net/service/request.go                                                     | 127 ++++++++++++++++++++++
 net/service/request.pb.go                                                  |  50 +++++++++
 net/service/request.proto                                                  |   6 ++
 net/service/request_test.go                                                |  41 +++++++
 net/service/service.go                                                     | 224 ++++++++++++++++++++++++++++++++++++++
 net/service/service_test.go                                                | 146 +++++++++++++++++++++++++
 net/swarm/conn.go                                                          | 210 ++++++++++++++++++++++++++++++++++++
 net/swarm/swarm.go                                                         | 190 +++++++++++++++++++++++++++++++++
 {swarm => net/swarm}/swarm_test.go                                         |  83 ++++++++------
 peer/peer.go                                                               |  35 ++++--
 peer/peer_test.go                                                          |   8 +-
 peer/peerstore.go                                                          |  87 +++++++++++++++
 peer/peerstore_test.go                                                     |  82 ++++++++++++++
 peer/queue/distance.go                                                     | 101 ++++++++++++++++++
 peer/queue/interface.go                                                    |  18 ++++
 peer/queue/queue_test.go                                                   | 131 +++++++++++++++++++++++
 peer/queue/sync.go                                                         |  74 +++++++++++++
 routing/dht/Makefile                                                       |  11 ++
 routing/dht/Message.go                                                     |  58 +++++-----
 routing/dht/dht.go                                                         | 712 +++++++++++++++++++++++++++++++++++++++++++++++--------------------------------------------------------------------------
 routing/dht/dht_logger.go                                                  |   5 +
 routing/dht/dht_test.go                                                    | 269 ++++++++++++++++++++++++++++------------------
 routing/dht/ext_test.go                                                    | 263 +++++++++++++++++++++++++--------------------
 routing/dht/handlers.go                                                    | 236 ++++++++++++++++++++++++++++++++++++++++
 routing/dht/messages.pb.go                                                 | 138 ++++++++++++------------
 routing/dht/messages.proto                                                 |  28 +++--
 routing/dht/query.go                                                       | 237 ++++++++++++++++++++++++++++++++++++++++
 routing/dht/routing.go                                                     | 422 ++++++++++++++++++++++++++++--------------------------------------------
 routing/kbucket/bucket.go                                                  |   2 +-
 routing/kbucket/table.go                                                   |   4 +-
 routing/kbucket/table_test.go                                              |   4 +-
 routing/kbucket/util.go                                                    |  54 ++--------
 routing/keyspace/keyspace.go                                               |  97 +++++++++++++++++
 routing/keyspace/xor.go                                                    |  74 +++++++++++++
 routing/keyspace/xor_test.go                                               | 147 +++++++++++++++++++++++++
 routing/routing.go                                                         |  13 +--
 swarm/interface.go                                                         |  20 ----
 swarm/mes_listener.go                                                      | 123 ---------------------
 swarm/mes_listener_test.go                                                 |  32 ------
 swarm/mes_wrapper.pb.go                                                    |  85 ---------------
 swarm/mes_wrapper.proto                                                    |  12 ---
 swarm/swarm.go                                                             | 506 --------------------------------------------------------------------------------------
 swarm/wrapper.go                                                           |  24 -----
 util/key_set.go                                                            |  46 ++++++++
 util/testutil/blocks.go                                                    |  22 ++++
 util/todocounter/counter.go                                                | 118 ++++++++++++++++++++
 util/util.go                                                               |  15 +++
 144 files changed, 9429 insertions(+), 2916 deletions(-)
 create mode 100644 Dockerfile
 create mode 100644 Godeps/_workspace/src/code.google.com/p/go.net/context/context.go
 create mode 100644 Godeps/_workspace/src/code.google.com/p/go.net/context/context_test.go
 create mode 100644 Godeps/_workspace/src/code.google.com/p/go.net/context/withtimeout_test.go
 create mode 100644 Godeps/_workspace/src/github.com/jbenet/go-multiaddr/net.go
 create mode 100644 Godeps/_workspace/src/github.com/jbenet/go-multiaddr/net_test.go
 create mode 100644 Godeps/_workspace/src/github.com/tuxychandru/pubsub/README.md
 create mode 100644 Godeps/_workspace/src/github.com/tuxychandru/pubsub/pubsub.go
 create mode 100644 Godeps/_workspace/src/github.com/tuxychandru/pubsub/pubsub_test.go
 delete mode 100644 bitswap/bitswap.go
 delete mode 100644 bitswap/ledger.go
 create mode 100644 blocks/blocks_test.go
 create mode 100644 blockstore/blockstore.go
 create mode 100644 blockstore/blockstore_test.go
 copy {cmd/ipfs => core/commands}/add.go (52%)
 create mode 100644 core/commands/cat.go
 create mode 100644 core/commands/ls.go
 create mode 100644 core/commands/pin.go
 create mode 100644 core/commands/refs.go
 create mode 100644 crypto/key_test.go
 create mode 100644 crypto/spipe/Makefile
 rename identify/identify.go => crypto/spipe/handshake.go (54%)
 rename {identify => crypto/spipe}/message.pb.go (52%)
 create mode 100644 crypto/spipe/message.proto
 create mode 100644 crypto/spipe/pipe.go
 create mode 100644 daemon/daemon.go
 create mode 100644 daemon/daemon_client.go
 create mode 100644 daemon/daemon_test.go
 create mode 100644 doc.go
 create mode 100644 exchange/bitswap/bitswap.go
 create mode 100644 exchange/bitswap/bitswap_test.go
 create mode 100644 exchange/bitswap/message/Makefile
 create mode 100644 exchange/bitswap/message/message.go
 rename {bitswap => exchange/bitswap/message}/message.pb.go (98%)
 rename {bitswap => exchange/bitswap/message}/message.proto (82%)
 create mode 100644 exchange/bitswap/message/message_test.go
 create mode 100644 exchange/bitswap/network/interface.go
 create mode 100644 exchange/bitswap/network/net_message_adapter.go
 create mode 100644 exchange/bitswap/notifications/notifications.go
 create mode 100644 exchange/bitswap/notifications/notifications_test.go
 create mode 100644 exchange/bitswap/strategy/interface.go
 create mode 100644 exchange/bitswap/strategy/ledger.go
 create mode 100644 exchange/bitswap/strategy/ledger_test.go
 rename bitswap/strategy.go => exchange/bitswap/strategy/math.go (55%)
 rename bitswap/strategy_test.go => exchange/bitswap/strategy/math_test.go (95%)
 create mode 100644 exchange/bitswap/strategy/strategy.go
 create mode 100644 exchange/bitswap/strategy/strategy_test.go
 create mode 100644 exchange/bitswap/testnet/network.go
 create mode 100644 exchange/bitswap/testnet/network_test.go
 create mode 100644 exchange/bitswap/testnet/routing.go
 create mode 100644 exchange/bitswap/testnet/routing_test.go
 create mode 100644 exchange/interface.go
 create mode 100644 exchange/offline/offline.go
 create mode 100644 exchange/offline/offline_test.go
 delete mode 100644 identify/identify_test.go
 delete mode 100644 identify/message.proto
 create mode 100644 importer/rabin.go
 delete mode 100644 importer/split_test.go
 rename {swarm => net/conn}/conn.go (66%)
 rename {swarm => net/conn}/conn_test.go (86%)
 create mode 100644 net/interface.go
 create mode 100644 net/message/message.go
 create mode 100644 net/mux/Makefile
 create mode 100644 net/mux/mux.go
 create mode 100644 net/mux/mux.pb.go
 create mode 100644 net/mux/mux.proto
 create mode 100644 net/mux/mux_test.go
 create mode 100644 net/net.go
 create mode 100644 net/net_test.go
 create mode 100644 net/service/Makefile
 create mode 100644 net/service/request.go
 create mode 100644 net/service/request.pb.go
 create mode 100644 net/service/request.proto
 create mode 100644 net/service/request_test.go
 create mode 100644 net/service/service.go
 create mode 100644 net/service/service_test.go
 create mode 100644 net/swarm/conn.go
 create mode 100644 net/swarm/swarm.go
 rename {swarm => net/swarm}/swarm_test.go (56%)
 create mode 100644 peer/peerstore.go
 create mode 100644 peer/peerstore_test.go
 create mode 100644 peer/queue/distance.go
 create mode 100644 peer/queue/interface.go
 create mode 100644 peer/queue/queue_test.go
 create mode 100644 peer/queue/sync.go
 create mode 100644 routing/dht/Makefile
 create mode 100644 routing/dht/handlers.go
 create mode 100644 routing/dht/query.go
 create mode 100644 routing/keyspace/keyspace.go
 create mode 100644 routing/keyspace/xor.go
 create mode 100644 routing/keyspace/xor_test.go
 delete mode 100644 swarm/interface.go
 delete mode 100644 swarm/mes_listener.go
 delete mode 100644 swarm/mes_listener_test.go
 delete mode 100644 swarm/mes_wrapper.pb.go
 delete mode 100644 swarm/mes_wrapper.proto
 delete mode 100644 swarm/swarm.go
 delete mode 100644 swarm/wrapper.go
 create mode 100644 util/key_set.go
 create mode 100644 util/testutil/blocks.go
 create mode 100644 util/todocounter/counter.go
First, rewinding head to replay your work on top of it...
Fast-forwarded master to 676d9c5c968c8ab839f8f3f0f41fc047eab86bc6.

@whyrusleeping

Copy link
Copy Markdown
Member

@jbenet o.o that changelist....

@btc
btc deleted the net branch September 23, 2014 21:09
@aschmahmann aschmahmann mentioned this pull request Sep 22, 2020
55 of 72 tasks
ariescodescream pushed a commit to ariescodescream/go-ipfs that referenced this pull request Oct 23, 2021
ariescodescream pushed a commit to ariescodescream/go-ipfs that referenced this pull request Apr 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants