Repository navigation
Fsync the directories a follower creates with the next fsync request - #2555
Merged
Merged
Conversation
|
No issues found. |
weistrand
marked this pull request as ready for review
October 26, 2026 16:20
viktorerlingsson
force-pushed
the
defer-follower-dir-fsyncs
branch
from
October 27, 2026 14:41
88cebc2 to
b23b462
Compare
kickster97
reviewed
Oct 27, 2026
kickster97
approved these changes
Oct 27, 2026
`FileSystem.mkdir_p` now takes a block that gets the parent of each directory it creates. The follower's `make_dirs` uses it to collect the parents for its next fsync request, instead of keeping its own copy of the directory walk.
A directory the follower creates for a replaced file is fsynced with the next fsync request, not before the replace is acked. The only such replace is a new queue's `.queue` file, which the leader doesn't fsync either.
A replace's new directories are fsynced with the next fsync request, and a syncfs, from a request without a path or a full sync, makes the next request skip them.
A v1 leader never sends fsync requests, so `@created_dirs` was only cleared on a full sync. Clear it in `send_ack_loop` before each syncfs instead, so a dir created while the syncfs blocks stays marked.
viktorerlingsson
force-pushed
the
defer-follower-dir-fsyncs
branch
from
October 27, 2026 16:07
b23b462 to
0f1e718
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
#2296 made a follower create directories with
FileSystem.mkdir_p, which fsyncs the parent of each directory it creates. That runs inline on the replication stream, so the first file of every new queue or MQTT session held up the follower's stream for an fsync. Declares wait for the followers, so on a cluster each new queue cost the leader that fsync's latency too. 2.10 created them without an fsync. Raised in the review of #2544.What
The follower creates directories without fsyncing them and remembers the parent of each one it creates. Those parents are fsynced with the next fsync request, before the ack that makes anything in them durable. A syncfs, from a v1 leader or after a full sync, covers them, so they're forgotten then.
A replace's ack doesn't cover a directory created for it. The only such replace is a new queue's
.queuefile, which the leader doesn't fsync either.FileSystem.mkdir_ptakes a block that gets each created parent, so the follower doesn't keep its own copy of it.Follower fsyncs counted with strace on a release build, a leader and one follower, 2,000 durable queues:
Each declare makes the leader request a fsync of
definitions.amqp, so the vhost directory is fsynced with that request, in the ack loop instead of on the stream. An import makes one request for all its queues, and they share a parent. The fsyncs left on the stream arereplace's fsync of.queue.tmp.How
make test