Skip to content

feat(p2p): add --foreground flag to listen and forward commands - #11099

Merged
lidel merged 10 commits into
masterfrom
feat/p2p-foreground
Jan 9, 2026
Merged

lidel merged 10 commits into
masterfrom
feat/p2p-foreground

Conversation

@lidel

@lidel lidel commented Dec 10, 2025 •

Copy link
Copy Markdown
Contributor

This PR mostly adds docs/p2p-tunnel.md documentation with useful examples.

It also

  • Closes p2p listen keep job running #11095 by adding -f/--foreground option that keeps the command running until interrupted (SIGTERM/Ctrl+C) or closed via ipfs p2p close. the listener/forwarder is automatically removed when the command exits. This is very useful for systemd services and scripts that need cleanup on exit.
  • Fixes race condition related to $IPFS_PATH/api|gateway files being created before goroutine that initializes listener is executed (80b703a)

@lidel lidel mentioned this pull request Dec 10, 2025
3 tasks done
adds `-f/--foreground` option that keeps the command running until
interrupted (SIGTERM/Ctrl+C) or closed via `ipfs p2p close`. the
listener/forwarder is automatically removed when the command exits.

useful for systemd services and scripts that need cleanup on exit.
@lidel
lidel force-pushed the feat/p2p-foreground branch from 34a8acc to e7bb812 Compare December 10, 2025 02:12
@Hazematman

Hazematman commented Dec 10, 2025 •

Copy link
Copy Markdown

Thanks! This works great for me I have two ipfs services, one to launch ipfs as a daemon and one to start forwarding ssh in the background.

ipfs.service

[Unit]
Description=IPFS daemon

[Service]
ExecStart=/usr/local/bin/ipfs daemon
Restart=on-failure

[Install]
WantedBy=default.target

ipfs-ssh-forward.service

[Unit]
Description=IPFS SSH forwarding
After=ipfs.service
Requires=ipfs.service

[Service]
ExecStartPre=/bin/sleep 10
ExecStart=/usr/local/bin/ipfs p2p listen /x/ssh /ip4/127.0.0.1/tcp/22 -f
Restart=on-failure
RestartSec=10

[Install]
WantedBy=default.target

This seems to work really nicely with systemd, the port listening will automatically restart with if for some reason that systemd service stops running and will relaunch with the ipfs service when it starts. Only quirk is I'm depending on delays to wait for ipfs daemon to fully startup before launching p2p listen. If there a file that gets created/modified when the daemon fully starts it could be tracked with one of the path directives in systemd (https://www.freedesktop.org/software/systemd/man/latest/systemd.path.html#PathExists=)

Can't really comment to much on the code to implement this as I'm not familiar with the code base or go-lang in general

EDIT:
I just realized there is the ~/.ipfs/repo.lock file I can use with PathExists= directive to wait until the daemon has launched.

@lidel

lidel commented Dec 10, 2025 •

Copy link
Copy Markdown
Contributor Author

@Hazematman you could also use ~/.ipfs/api ($IPFS_PATH/api) in PathExists it is the address of RPC API ipfs p2p CLI will use to connect to the daemon. could be more appropriate. lmk if it works better/worse (+ mind commenting with updated examples of your systemd scripts? I'd like to include them in docs if possible)

@Hazematman

Hazematman commented Dec 11, 2025 •

Copy link
Copy Markdown

I'm still testing it for robustness but this is what I have now for systemd services

I have a ipfs-ssh-forward.service unit file that looks like

[Unit]
Description=IPFS SSH forwarding
Requires=ipfs.service

[Service]
ExecStart=/usr/local/bin/ipfs p2p listen /x/ssh /ip4/127.0.0.1/tcp/22 -f
Restart=on-failure
RestartSec=10

[Install]
WantedBy=default.target

Then I have a ipfs-ssh-forward.path systemd path file

[Unit]
Description=Monitor for start of ipfs
After=ipfs.service
Requires=ipfs.service

[Path]
PathExists=<home>/.ipfs/api
PathExists=<home>/.ipfs/repo.lock
Unit=ipfs-ssh-forward.service

[Install]
WantedBy=default.target

I'm using the ipfs.service basically unmodified described here https://github.com/ipfs/kubo/tree/master/misc

Then to enable things I do

systemctl --user enable ipfs.service
systemctl --user enable ipfs-ssh-forward.path
systemctl --user start ipfs.service

The ipfs service should startup and then once the repo.lock and api files both exist the ipfs-ssh-forward.service is started. I've tested kill the and restarting the ipfs service, closing ipfs p2p connection with ipfs p2p close -a, and restarting the machine a few times and this seems to reliable ensure that the p2p listen is always running.

The one weird thing I'm trying to debug is that it seems that the first launch of the ipfs-ssh-forward service always fails. I'm not 100% sure why but one theory is that when the api file is created, ipfs isn't ready for p2p listen commands yet.

- add dedicated docs/p2p-tunnels.md covering:
  - why p2p tunnels (NAT traversal, no public IP needed)
  - quick start with netcat
  - background and foreground modes
  - systemd integration with path-based activation
  - security considerations and troubleshooting
- document Experimental.Libp2pStreamMounting in docs/config.md
- simplify docs/experimental-features.md, link to new doc
- add "Learn more" links to ipfs p2p listen/forward --help
- update changelog entry with doc link
- add cross-reference in misc/README.md
fixes race condition where $IPFS_PATH/api and $IPFS_PATH/gateway files
were written before the HTTP servers were ready to accept connections.
this caused issues for tools like systemd path units that immediately
try to connect when these files appear.

changes:
- add corehttp.ServeWithReady() that signals when server is ready
- wait for ready signal before writing address files
- use sync.WaitGroup.Go() (Go 1.25) for cleaner goroutine management
- add TestAddressFileReady to verify both api and gateway files
@lidel

lidel commented Dec 16, 2025

Copy link
Copy Markdown
Contributor Author

Thank you @Hazematman, there was indeed a race condition related to creation of $IPFS_PATH/api file.
Pushed fix in 80b703a, try the latest version of this PR if it makes any difference.

Also, check docs/p2p-tunnels.md – wanted to document this feature in more friendly way with examples, incorporated your systemd examples – lmk if this looks ok to you.

If this PR looks sensible, we will include this in 0.40.

@lidel lidel mentioned this pull request Dec 16, 2025
49 of 67 tasks
@lidel
lidel marked this pull request as ready for review December 16, 2025 23:39
@lidel
lidel requested a review from a team as a code owner December 16, 2025 23:39
@lidel lidel added the status/ready Ready to be worked label Dec 17, 2025
Comment thread cmd/ipfs/kubo/daemon.go Resolved
Comment thread cmd/ipfs/kubo/daemon.go Resolved
Comment thread cmd/ipfs/kubo/daemon.go Resolved
- buffer error channel with len(listeners) to prevent deadlock when
  multiple servers write errors simultaneously
- wait for ALL listeners to be ready before writing api/gateway file,
  not just the first one

Feedback-from: #11099 (review)
reframe to lead with user benefit and add example output
Comment thread docs/p2p-tunnels.md Resolved
@Hazematman

Copy link
Copy Markdown

With 80b703a I'm not seeing the startup race condition I saw before with the same tests I did restarting my machine.

The docs/p2p-tunnels.md also looks good to me. I had one comment about where you mentioned auto restarting systemd as its useful not just for the startup race condition issue.

@lidel lidel self-assigned this Jan 9, 2026
lidel added 2 commits January 9, 2026 18:43
the "First launch fails but restarts work" troubleshooting section
described a race where the api file was written before the daemon was
ready. this was fixed in 80b703a which ensures api/gateway files are
only written after HTTP servers are ready to accept connections.
@lidel
lidel merged commit 25ebab9 into master Jan 9, 2026
20 checks passed
@lidel
lidel deleted the feat/p2p-foreground branch January 9, 2026 18:22
@lidel lidel added the iteration/2026-q1 On maintainer radar for Q1 2026 label Feb 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

iteration/2026-q1 On maintainer radar for Q1 2026 status/ready Ready to be worked

Projects

None yet

Development

Successfully merging this pull request may close these issues.

p2p listen keep job running

3 participants