Skip to content

peerstore: fix addressbook benchmark timing - #3092

Merged
sukunrt merged 1 commit into
masterfrom
sukun/peerstore-bms
Dec 17, 2024
Merged

sukunrt merged 1 commit into
masterfrom
sukun/peerstore-bms

Conversation

@sukunrt

@sukunrt sukunrt commented Dec 8, 2024

Copy link
Copy Markdown
Member

The previous code was not accounting accurately for the time spent generating test cases. In the benchmark output below the difference between 100 address per peer suite is huge:

BenchmarkInMemoryPeerstore/SetAddrs-100Addrs-InMem-16              10000	    190198 ns/op
vs
BenchmarkInMemoryPeerstore/SetAddrs-100-16                	   10000	     42710 ns/op

old benchmark:

goos: linux
goarch: amd64
pkg: github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoremem
cpu: AMD Ryzen 7 7840U w/ Radeon  780M Graphics
BenchmarkInMemoryPeerstore
BenchmarkInMemoryPeerstore/AddAddrs-1Addrs-InMem
BenchmarkInMemoryPeerstore/AddAddrs-1Addrs-InMem-16         	   10000	      4158 ns/op
BenchmarkInMemoryPeerstore/AddAddrs-10Addrs-InMem
BenchmarkInMemoryPeerstore/AddAddrs-10Addrs-InMem-16        	   10000	     26364 ns/op
BenchmarkInMemoryPeerstore/AddAddrs-100Addrs-InMem
BenchmarkInMemoryPeerstore/AddAddrs-100Addrs-InMem-16       	   10000	    197692 ns/op
BenchmarkInMemoryPeerstore/AddGetAndClearAddrs-1Addrs-InMem
BenchmarkInMemoryPeerstore/AddGetAndClearAddrs-1Addrs-InMem-16         	   10000	      3692 ns/op
BenchmarkInMemoryPeerstore/AddGetAndClearAddrs-10Addrs-InMem
BenchmarkInMemoryPeerstore/AddGetAndClearAddrs-10Addrs-InMem-16        	   10000	     23182 ns/op
BenchmarkInMemoryPeerstore/AddGetAndClearAddrs-100Addrs-InMem
BenchmarkInMemoryPeerstore/AddGetAndClearAddrs-100Addrs-InMem-16       	   10000	    202959 ns/op
BenchmarkInMemoryPeerstore/Get1000PeersWithAddrs-1Addrs-InMem
BenchmarkInMemoryPeerstore/Get1000PeersWithAddrs-1Addrs-InMem-16       	   10000	     25348 ns/op
BenchmarkInMemoryPeerstore/Get1000PeersWithAddrs-10Addrs-InMem
BenchmarkInMemoryPeerstore/Get1000PeersWithAddrs-10Addrs-InMem-16      	   10000	     20133 ns/op
BenchmarkInMemoryPeerstore/Get1000PeersWithAddrs-100Addrs-InMem
BenchmarkInMemoryPeerstore/Get1000PeersWithAddrs-100Addrs-InMem-16     	   10000	     17525 ns/op
BenchmarkInMemoryPeerstore/GetAddrs-1Addrs-InMem
BenchmarkInMemoryPeerstore/GetAddrs-1Addrs-InMem-16                    	   10000	       163.5 ns/op
BenchmarkInMemoryPeerstore/GetAddrs-10Addrs-InMem
BenchmarkInMemoryPeerstore/GetAddrs-10Addrs-InMem-16                   	   10000	       203.9 ns/op
BenchmarkInMemoryPeerstore/GetAddrs-100Addrs-InMem
BenchmarkInMemoryPeerstore/GetAddrs-100Addrs-InMem-16                  	   10000	      1696 ns/op
BenchmarkInMemoryPeerstore/SetAddrs-1Addrs-InMem
BenchmarkInMemoryPeerstore/SetAddrs-1Addrs-InMem-16                    	   10000	      3431 ns/op
BenchmarkInMemoryPeerstore/SetAddrs-10Addrs-InMem
BenchmarkInMemoryPeerstore/SetAddrs-10Addrs-InMem-16                   	   10000	     21599 ns/op
BenchmarkInMemoryPeerstore/SetAddrs-100Addrs-InMem
BenchmarkInMemoryPeerstore/SetAddrs-100Addrs-InMem-16                  	   10000	    190198 ns/op

new benchmark:

goos: linux
goarch: amd64
pkg: github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoremem
cpu: AMD Ryzen 7 7840U w/ Radeon  780M Graphics
BenchmarkInMemoryPeerstore
BenchmarkInMemoryPeerstore/AddAddrs-1
BenchmarkInMemoryPeerstore/AddAddrs-1-16         	   10000	       899.2 ns/op
BenchmarkInMemoryPeerstore/GetAddrs-1
BenchmarkInMemoryPeerstore/GetAddrs-1-16         	   10000	       914.9 ns/op
BenchmarkInMemoryPeerstore/GetAndClearAddrs-1
BenchmarkInMemoryPeerstore/GetAndClearAddrs-1-16 	   10000	       699.5 ns/op
BenchmarkInMemoryPeerstore/PeersWithAddrs-1
BenchmarkInMemoryPeerstore/PeersWithAddrs-1-16   	   10000	    112298 ns/op
BenchmarkInMemoryPeerstore/SetAddrs-1
BenchmarkInMemoryPeerstore/SetAddrs-1-16         	   10000	       764.7 ns/op
BenchmarkInMemoryPeerstore/AddAddrs-10
BenchmarkInMemoryPeerstore/AddAddrs-10-16        	   10000	      4682 ns/op
BenchmarkInMemoryPeerstore/GetAddrs-10
BenchmarkInMemoryPeerstore/GetAddrs-10-16        	   10000	      5310 ns/op
BenchmarkInMemoryPeerstore/GetAndClearAddrs-10
BenchmarkInMemoryPeerstore/GetAndClearAddrs-10-16         	   10000	      4147 ns/op
BenchmarkInMemoryPeerstore/PeersWithAddrs-10
BenchmarkInMemoryPeerstore/PeersWithAddrs-10-16           	   10000	     93408 ns/op
BenchmarkInMemoryPeerstore/SetAddrs-10
BenchmarkInMemoryPeerstore/SetAddrs-10-16                 	   10000	      4412 ns/op
BenchmarkInMemoryPeerstore/AddAddrs-100
BenchmarkInMemoryPeerstore/AddAddrs-100-16                	   10000	     40847 ns/op
BenchmarkInMemoryPeerstore/GetAddrs-100
BenchmarkInMemoryPeerstore/GetAddrs-100-16                	   10000	     40197 ns/op
BenchmarkInMemoryPeerstore/GetAndClearAddrs-100
BenchmarkInMemoryPeerstore/GetAndClearAddrs-100-16        	   10000	     37848 ns/op
BenchmarkInMemoryPeerstore/PeersWithAddrs-100
BenchmarkInMemoryPeerstore/PeersWithAddrs-100-16          	   10000	    111613 ns/op
BenchmarkInMemoryPeerstore/SetAddrs-100
BenchmarkInMemoryPeerstore/SetAddrs-100-16                	   10000	     42710 ns/op

The previous code was not accounting accurately for the time spent
generating test cases. In the benchmark output below the difference
between 100 address per peer suite is huge:

BenchmarkInMemoryPeerstore/SetAddrs-100Addrs-InMem-16              10000	    190198 ns/op
vs
BenchmarkInMemoryPeerstore/SetAddrs-100-16                	   10000	     42710 ns/op

old benchmark:

```
goos: linux
goarch: amd64
pkg: github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoremem
cpu: AMD Ryzen 7 7840U w/ Radeon  780M Graphics
BenchmarkInMemoryPeerstore
BenchmarkInMemoryPeerstore/AddAddrs-1Addrs-InMem
BenchmarkInMemoryPeerstore/AddAddrs-1Addrs-InMem-16         	   10000	      4158 ns/op
BenchmarkInMemoryPeerstore/AddAddrs-10Addrs-InMem
BenchmarkInMemoryPeerstore/AddAddrs-10Addrs-InMem-16        	   10000	     26364 ns/op
BenchmarkInMemoryPeerstore/AddAddrs-100Addrs-InMem
BenchmarkInMemoryPeerstore/AddAddrs-100Addrs-InMem-16       	   10000	    197692 ns/op
BenchmarkInMemoryPeerstore/AddGetAndClearAddrs-1Addrs-InMem
BenchmarkInMemoryPeerstore/AddGetAndClearAddrs-1Addrs-InMem-16         	   10000	      3692 ns/op
BenchmarkInMemoryPeerstore/AddGetAndClearAddrs-10Addrs-InMem
BenchmarkInMemoryPeerstore/AddGetAndClearAddrs-10Addrs-InMem-16        	   10000	     23182 ns/op
BenchmarkInMemoryPeerstore/AddGetAndClearAddrs-100Addrs-InMem
BenchmarkInMemoryPeerstore/AddGetAndClearAddrs-100Addrs-InMem-16       	   10000	    202959 ns/op
BenchmarkInMemoryPeerstore/Get1000PeersWithAddrs-1Addrs-InMem
BenchmarkInMemoryPeerstore/Get1000PeersWithAddrs-1Addrs-InMem-16       	   10000	     25348 ns/op
BenchmarkInMemoryPeerstore/Get1000PeersWithAddrs-10Addrs-InMem
BenchmarkInMemoryPeerstore/Get1000PeersWithAddrs-10Addrs-InMem-16      	   10000	     20133 ns/op
BenchmarkInMemoryPeerstore/Get1000PeersWithAddrs-100Addrs-InMem
BenchmarkInMemoryPeerstore/Get1000PeersWithAddrs-100Addrs-InMem-16     	   10000	     17525 ns/op
BenchmarkInMemoryPeerstore/GetAddrs-1Addrs-InMem
BenchmarkInMemoryPeerstore/GetAddrs-1Addrs-InMem-16                    	   10000	       163.5 ns/op
BenchmarkInMemoryPeerstore/GetAddrs-10Addrs-InMem
BenchmarkInMemoryPeerstore/GetAddrs-10Addrs-InMem-16                   	   10000	       203.9 ns/op
BenchmarkInMemoryPeerstore/GetAddrs-100Addrs-InMem
BenchmarkInMemoryPeerstore/GetAddrs-100Addrs-InMem-16                  	   10000	      1696 ns/op
BenchmarkInMemoryPeerstore/SetAddrs-1Addrs-InMem
BenchmarkInMemoryPeerstore/SetAddrs-1Addrs-InMem-16                    	   10000	      3431 ns/op
BenchmarkInMemoryPeerstore/SetAddrs-10Addrs-InMem
BenchmarkInMemoryPeerstore/SetAddrs-10Addrs-InMem-16                   	   10000	     21599 ns/op
BenchmarkInMemoryPeerstore/SetAddrs-100Addrs-InMem
BenchmarkInMemoryPeerstore/SetAddrs-100Addrs-InMem-16                  	   10000	    190198 ns/op
```

new benchmark:
```
goos: linux
goarch: amd64
pkg: github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoremem
cpu: AMD Ryzen 7 7840U w/ Radeon  780M Graphics
BenchmarkInMemoryPeerstore
BenchmarkInMemoryPeerstore/AddAddrs-1
BenchmarkInMemoryPeerstore/AddAddrs-1-16         	   10000	       899.2 ns/op
BenchmarkInMemoryPeerstore/GetAddrs-1
BenchmarkInMemoryPeerstore/GetAddrs-1-16         	   10000	       914.9 ns/op
BenchmarkInMemoryPeerstore/GetAndClearAddrs-1
BenchmarkInMemoryPeerstore/GetAndClearAddrs-1-16 	   10000	       699.5 ns/op
BenchmarkInMemoryPeerstore/PeersWithAddrs-1
BenchmarkInMemoryPeerstore/PeersWithAddrs-1-16   	   10000	    112298 ns/op
BenchmarkInMemoryPeerstore/SetAddrs-1
BenchmarkInMemoryPeerstore/SetAddrs-1-16         	   10000	       764.7 ns/op
BenchmarkInMemoryPeerstore/AddAddrs-10
BenchmarkInMemoryPeerstore/AddAddrs-10-16        	   10000	      4682 ns/op
BenchmarkInMemoryPeerstore/GetAddrs-10
BenchmarkInMemoryPeerstore/GetAddrs-10-16        	   10000	      5310 ns/op
BenchmarkInMemoryPeerstore/GetAndClearAddrs-10
BenchmarkInMemoryPeerstore/GetAndClearAddrs-10-16         	   10000	      4147 ns/op
BenchmarkInMemoryPeerstore/PeersWithAddrs-10
BenchmarkInMemoryPeerstore/PeersWithAddrs-10-16           	   10000	     93408 ns/op
BenchmarkInMemoryPeerstore/SetAddrs-10
BenchmarkInMemoryPeerstore/SetAddrs-10-16                 	   10000	      4412 ns/op
BenchmarkInMemoryPeerstore/AddAddrs-100
BenchmarkInMemoryPeerstore/AddAddrs-100-16                	   10000	     40847 ns/op
BenchmarkInMemoryPeerstore/GetAddrs-100
BenchmarkInMemoryPeerstore/GetAddrs-100-16                	   10000	     40197 ns/op
BenchmarkInMemoryPeerstore/GetAndClearAddrs-100
BenchmarkInMemoryPeerstore/GetAndClearAddrs-100-16        	   10000	     37848 ns/op
BenchmarkInMemoryPeerstore/PeersWithAddrs-100
BenchmarkInMemoryPeerstore/PeersWithAddrs-100-16          	   10000	    111613 ns/op
BenchmarkInMemoryPeerstore/SetAddrs-100
BenchmarkInMemoryPeerstore/SetAddrs-100-16                	   10000	     42710 ns/op
```
@sukunrt
sukunrt requested a review from MarcoPolo December 8, 2024 07:50
@sukunrt sukunrt self-assigned this Dec 9, 2024
@p-shahi p-shahi mentioned this pull request Dec 16, 2024
10 of 26 tasks
@sukunrt
sukunrt merged commit ba1703f into master Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants