Repository navigation
Description
Problem
At Shipyard we've run some A/B tests at public gateway and lowering nginx timeout from 5m to 30s.
This produced better UX and also raised the number of 200s while lowering 504s.
Right now, the boxo/gateway library does not have any timeout, aside from this failsafe 1h one, so we set timeout at .nginx is sitting in front of rainbow.
This is extra step that most of people running gateways does not do, thus wasting resources while looking for content that is not provided correctly to certain degree (nginx default timeout is 60s, while it could be lowered).
What is really unfortunate is that IPFS Desktop users hit gateway directly, and they never hit any timeout, unless it is their user agent (browser).
Proposed feature
We should introduce feature similar to nginx's proxy_read_timeout directly in the boxo/gateway library, make it configurable, but also set it to some implicit default (e.g. 30s).
It should not depend on any internal gateway logic, but solely count the time between two successful writes from server to the client.
This way everyone using boxo will save resources, and Desktop users will get meaningful error page sooner, and we will not regress.
Implementation ideas
Details tbd, but broad strokes idea for the boxo/gateway library will be to wrap existing handler in a generic response writer timeout handler:
func main() {
gwHandler := // current boxo/gateway handler
timeoutHandler := WithResponseWriteTimeout(gwHandler, 30*time.Second) // future handler will act like this
http.ListenAndServe(":8080", timeoutHandler)
}- The
WithResponseWriteTimeoutmiddleware creates a timeoutResponseWriter and starts a timer.- "timeoutResponseWriter" wraps the original
ResponseWriterand tracks the last successful write. - Every time data is written successfully, the timer is reset.
- If no data is written for the specified duration, the timer expires, and a 504 Gateway Timeout status is sent to the client.
- "timeoutResponseWriter" wraps the original
Configuration-wise, Config struct would get time.Duration field similar to block timeout in backend here, and NewHandler(config, backend) would set implicit default if not provided in config.
- We should check context for list of things that node tried, and print useful error to user (i tried routing, found no peers, or found 4 pers, but all offline)
Activity
[-]gateway: implicit write timeout[/-][+]gateway: ability to set response write timeout[/+]gammazero commentedon Jan 28, 2025
I think that a single timeout should cover all writes in the response. Otherwise, a trickle of data, byte-by-byte, could take a very long time.
If this were a timeout waiting to read a response then returning
http.StatusGatewayTimeoutis appropriate. Since this is a write timeout, I thinkhttp.StatusServiceUnavailableis preferable. This is the behavior ofhttp.TimeoutHandlerfeat(gw): concurrency and timeout limits
lidel commentedon Aug 11, 2025
I think a better name for what we need is "Retrieval Timeout" that covers both:
I took a stab at implementing/documenting this approach in #994
feat(gw): concurrency and timeout limits
feat(gateway): concurrency and retrieval timeout limits (#994)