You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We use the Jetty HTTP client's async API and want it to shut down gracefully as part of our application's shutdown sequence, completing any in-progress requests.
Calling HttpClient.stop() aborts all the pending requests immediately, so we have to wait for requests to finish before calling it. There doesn't seem to be a documented API to query a client to see if it has any active requests. We've implemented a hacked-up solution by poking around the internal states of the HTTP client's various child objects. That approach is likely to break when we upgrade to new Jetty releases. A public API to either shut down gracefully or to query whether there are any outstanding requests would be far preferable.
Note you have to iterate on all destinations. @sbordet WDYT about adding a method for such use case? what about methods similar to ExecutorService such shutdownNow() and shutdown()?
Maybe a bit of a tangent, but checking for exchanges on the destination was one of the first things I tried and I couldn't get it to work. Here's a simple test case that fails for me.
@Test
public void testHttpExchangesWithOpenConnection() throws Exception {
HttpClient client = new HttpClient();
ServerSocket serverSocket = new ServerSocket(0);
client.start();
Request request = client.newRequest("http://localhost:" + serverSocket.getLocalPort());
request.send(result -> System.out.println("Complete"));
serverSocket.accept();
Thread.sleep(500); // test passes if this is commented out
boolean hasAnyExchanges = client.getDestinations().stream()
.map(dest -> ((HttpDestination) dest).getHttpExchanges())
.anyMatch(exchanges -> !exchanges.isEmpty());
// A connection is open and the server hasn't returned a response yet.
assertTrue(hasAnyExchanges);
}
actually it's fine :-) As you don't send back any content/response the client never go to response listener callback so you cannot see System.out.println("Complete")
So here everything normal the http exchange has been finished..
If I were to shut down as soon as the exchange lists were all empty, I would be closing the connection during the middle of an active request. Maybe not a problem for a simple query or an idempotent operation, but imagine the request causes the server to take some real-world action like sending email. I'd never find out if the operation succeeded or returned an error of some kind. In an ideal world the server would give me a query API to find out the status of the operation after an unexpected connection close, but a lot of third-party services don't support that.
My workaround is not wonderful, but gets the job done. In my initialization, I call httpClient.setRemoveIdleDestinations(true); and then in the application's shutdown code, after the parts of the application that can initiate new requests have been shut down:
// The client is set to remove destinations that aren't in use, so the list of destinations
// can be used as a proxy for the list of active connections. But if there are connection
// errors, the client does not remove the destinations, so only wait for up to the maximum
// request idle time. This may erroneously shut down a request that's trickling a response
// back to us slowly.
if (!httpClient.getDestinations().isEmpty()) {
log.info("Waiting for outgoing HTTP requests to complete");
long timeout = System.currentTimeMillis() + IDLE_TIME_MS;
boolean foundConnection;
do {
Thread.sleep(100);
foundConnection = httpClient.getDestinations().stream()
.map(Destination::getHost)
.anyMatch(host -> !"null".equals(host)); // yes, it uses the string "null"
} while (foundConnection && System.currentTimeMillis() < timeout);
if (foundConnection) {
log.warn("Timed out waiting for outgoing HTTP requests to finish");
} else {
log.info("Outgoing HTTP requests finished");
}
}
httpClient.stop();
I'm not sure I want to add this feature to HttpClient. On the other hand it can be easily implemented in application code.
The idea is to guard sending of the requests until the application wants to shut it down. A boolean does the job, and when flipped it will just forbid further request sending.
As for waiting that sent requests complete, this is easily achieved using a Phaser, by calling register() when a request is sent (easily done from a Request.QueuedListener), and by calling arrive() from a Response.CompleteListener.
An alternative to Phaser could be an AtomicLong plus some waiting functionality.
Once the Phaser completes, you can call HttpClient.stop().
For what it's worth, I initially considered something like what you suggest (keeping track of the number of requests) but wasn't crazy about the idea because it means that all the places in the application where HTTP requests are initiated have to make sure to add listeners to increment and decrement the count reliably; if a developer, say, adds a code path in a CompleteListener that can cause it to return without calling arrive(), the shutdown code will think the request is still in flight.
That risk can be lowered by forbidding direct use of the request-sending API methods and writing some wrappers that add listeners, of course. But the HTTP client already knows exactly which requests are in what states and its list will always be 100% accurate even if the application code has bugs.
Obviously, though, my perspective on what changes make sense to include in the library is pretty limited, so no problem if this isn't something worth considering.
I second @sgrimm-sg's point: implementing this in application code is not particularly trivial. Seeing as Jetty already has this information on-hand it would be great if it could use it (provide a graceful shutdown method) or expose it (so we can use it to implement graceful shutdown ourselves).
In my own code, there is a strong separation between the factory that creates, starts, stops a HttpClient and code that uses it to initiate requests, parse responses. Ideally, I don't want to mix the two across they are logically far from each other (separation of concerns).
@sbordet Regarding your suggestion, how would you handle new WebsocketClient(HttpClient)? I can't listen to Request.QueuedListener, Response.CompleteListener in that case.
There are a ton of issues to take care of to implement this feature in a generic way.
Having the application know what they do will make much easier for applications to implement graceful shutdown.
Issues to consider in case a "shutdown" is triggered:
What happens to requests sent after shutdown?
What happens to requests queued before shutdown?
What happens to conversations such as redirects or authentications if shutdown happens in the middle of the conversation?
What happens to idle requests/responses that are not complete? Wait for idle timeout or not?
What if your application logic implements a retry mechanism for failed requests?
The notion of "graceful" is application dependent: some would like in-flight requests to finish, but won't wait for the idle timeout to expire, some don't care about redirects, but care about authentications, etc., etc.
Making everybody happy will be very difficult, but we're open to suggestions for a truly generic solution.
@sbordet As an application developer, I think the following functionality would be useful (though obviously this list is not all-inclusive):
Ability to get the number of outstanding requests, and maybe even get the Request objects themselves (for troubleshooting purposes)
Ability to wait for the outstanding requests to hit 0, with a timeout. Let the application figure out what happens if a timeout occurs (that's out of scope for you).
Ability to forcibly close all outstanding requests.
Ability to mark the HttpClient in "shutdown in progress" mode. While it is in this mode, attempting to create new requests will throw an exception.
For my application, the big issue is whether or not a request has been sent to the remote host. If so, I need to wait for the request to finish or time out. If not, it is safe to immediately abort since the request can't have caused the remote host to perform any potentially non-idempotent actions.
I agree with everything @cowwoc said above. To answer the specific questions (which I realize are not an exhaustive list):
Requests sent after shutdown: Throw an exception.
Requests queued before shutdown: Call their listeners with an exception indicating the requests were aborted due to shutdown.
Conversations such as redirects or authentications: Continue the requests until they complete. The library can't know if the original request that triggered the redirect or auth-needed response caused the server to take action. If the application cares about controlling this with finer granularity, it can tell the library to not automatically follow redirects, etc.
Idle requests/responses: Wait for them to finish or time out. The application can call the non-graceful shutdown method if it decides it doesn't want to wait any more.
Application retry logic: This isn't something I'd expect the HTTP client library to even attempt to address; it's up to the application to know if it's shutting down or not.
In general I agree that it is impossible to come up with a single answer that will be completely correct for every application. But I think as long as the behavior is well-defined and supports waiting for in-progress things to finish, the exact definition of "in-progress" (which is what most of the questions are about) is less critical. For example, though it wouldn't be completely ideal for my application, I'd be satisfied if queued requests were processed rather than aborted or if requests could continue to be sent after shutdown.
@sgrimm-sg One could potentially allow queued requests to continue by default and have a separate method that explicitly cancels all queued requests. That should satisfy both possibilities.
@sbordet I tried implementing your suggestion, keeping track of connections using Request.QueuedListener, Response.CompleteListener (I'm actually using BufferingResponseListener.onComplete for the latter)
It turns out that onComplete is invoked with a Result whose embedded Request object is not equal to the Request object that initiated the request (the URL and HTTP method are the same but the Request object is not). This means my code has no way of knowing that a previously-fired request has actually completed.
I debugged this further. If I understand correctly, It looks like the original request is redirected by the server. This leads to a new Request object getting created and that's the one I eventually receive.
I need some sort of event listener to let me know about the disposal of the original Request object and the creation of the new one. Alternatively, I need a way of knowing that the first request is related to the last request. (The only piece of information I found to link the two is HttpRequest.getConversation() but the method is protected). Comparing the URL, HTTP method is not enough because my client fires repeated requests on a regular basis.
As far as I can tell, Jetty does not provide enough information to do either of these things :(
I am really hoping I don't have to disable redirects and handle them myself because my code is already very hairy doing all this connection tracking. I am trying desperately to hide these low-level details from my application code.
I ended up using reflection to read HttpRequest.getConversation(). This allowed me to map Result.getRequest() back to the original Request object. It's an ugly hack with poor performance, but it works :)
Please let me know once you guys come up with something I can use long-term solution.
This issue has been automatically marked as stale because it has been a
full year without activity. It will be closed if no further activity occurs.
Thank you for your contributions.
Activity
olamy commentedon Apr 4, 2017
You can use something such
Note you have to iterate on all destinations.
@sbordet WDYT about adding a method for such use case? what about methods similar to ExecutorService such shutdownNow() and shutdown()?
sgrimm-sg commentedon Apr 4, 2017
Maybe a bit of a tangent, but checking for exchanges on the destination was one of the first things I tried and I couldn't get it to work. Here's a simple test case that fails for me.
olamy commentedon Apr 11, 2017
actually it's fine :-) As you don't send back any content/response the client never go to response listener callback so you cannot see
System.out.println("Complete")So here everything normal the http exchange has been finished..
sgrimm-sg commentedon Apr 11, 2017
If I were to shut down as soon as the exchange lists were all empty, I would be closing the connection during the middle of an active request. Maybe not a problem for a simple query or an idempotent operation, but imagine the request causes the server to take some real-world action like sending email. I'd never find out if the operation succeeded or returned an error of some kind. In an ideal world the server would give me a query API to find out the status of the operation after an unexpected connection close, but a lot of third-party services don't support that.
My workaround is not wonderful, but gets the job done. In my initialization, I call
httpClient.setRemoveIdleDestinations(true);and then in the application's shutdown code, after the parts of the application that can initiate new requests have been shut down:sbordet commentedon Apr 11, 2017
I'm not sure I want to add this feature to
HttpClient. On the other hand it can be easily implemented in application code.The idea is to guard sending of the requests until the application wants to shut it down. A
booleandoes the job, and when flipped it will just forbid further request sending.As for waiting that sent requests complete, this is easily achieved using a
Phaser, by callingregister()when a request is sent (easily done from aRequest.QueuedListener), and by callingarrive()from aResponse.CompleteListener.An alternative to
Phasercould be anAtomicLongplus some waiting functionality.Once the
Phasercompletes, you can callHttpClient.stop().sgrimm-sg commentedon Apr 12, 2017
For what it's worth, I initially considered something like what you suggest (keeping track of the number of requests) but wasn't crazy about the idea because it means that all the places in the application where HTTP requests are initiated have to make sure to add listeners to increment and decrement the count reliably; if a developer, say, adds a code path in a
CompleteListenerthat can cause it to return without callingarrive(), the shutdown code will think the request is still in flight.That risk can be lowered by forbidding direct use of the request-sending API methods and writing some wrappers that add listeners, of course. But the HTTP client already knows exactly which requests are in what states and its list will always be 100% accurate even if the application code has bugs.
Obviously, though, my perspective on what changes make sense to include in the library is pretty limited, so no problem if this isn't something worth considering.
olamy commentedon Nov 10, 2017
IMHO can be close
cowwoc commentedon Jul 27, 2018
I second @sgrimm-sg's point: implementing this in application code is not particularly trivial. Seeing as Jetty already has this information on-hand it would be great if it could use it (provide a graceful shutdown method) or expose it (so we can use it to implement graceful shutdown ourselves).
In my own code, there is a strong separation between the factory that creates, starts, stops a
HttpClientand code that uses it to initiate requests, parse responses. Ideally, I don't want to mix the two across they are logically far from each other (separation of concerns).@sbordet Regarding your suggestion, how would you handle
new WebsocketClient(HttpClient)? I can't listen toRequest.QueuedListener,Response.CompleteListenerin that case.sbordet commentedon Jul 29, 2018
There are a ton of issues to take care of to implement this feature in a generic way.
Having the application know what they do will make much easier for applications to implement graceful shutdown.
Issues to consider in case a "shutdown" is triggered:
The notion of "graceful" is application dependent: some would like in-flight requests to finish, but won't wait for the idle timeout to expire, some don't care about redirects, but care about authentications, etc., etc.
Making everybody happy will be very difficult, but we're open to suggestions for a truly generic solution.
cowwoc commentedon Jul 29, 2018
@sbordet As an application developer, I think the following functionality would be useful (though obviously this list is not all-inclusive):
HttpClientin "shutdown in progress" mode. While it is in this mode, attempting to create new requests will throw an exception.sgrimm-sg commentedon Jul 30, 2018
For my application, the big issue is whether or not a request has been sent to the remote host. If so, I need to wait for the request to finish or time out. If not, it is safe to immediately abort since the request can't have caused the remote host to perform any potentially non-idempotent actions.
I agree with everything @cowwoc said above. To answer the specific questions (which I realize are not an exhaustive list):
In general I agree that it is impossible to come up with a single answer that will be completely correct for every application. But I think as long as the behavior is well-defined and supports waiting for in-progress things to finish, the exact definition of "in-progress" (which is what most of the questions are about) is less critical. For example, though it wouldn't be completely ideal for my application, I'd be satisfied if queued requests were processed rather than aborted or if requests could continue to be sent after shutdown.
cowwoc commentedon Jul 30, 2018
@sgrimm-sg One could potentially allow queued requests to continue by default and have a separate method that explicitly cancels all queued requests. That should satisfy both possibilities.
cowwoc commentedon Aug 2, 2018
@sbordet I tried implementing your suggestion, keeping track of connections using
Request.QueuedListener,Response.CompleteListener(I'm actually usingBufferingResponseListener.onCompletefor the latter)It turns out that
onCompleteis invoked with aResultwhose embeddedRequestobject is not equal to theRequestobject that initiated the request (the URL and HTTP method are the same but theRequestobject is not). This means my code has no way of knowing that a previously-fired request has actually completed.Any suggestions?
cowwoc commentedon Aug 2, 2018
I debugged this further. If I understand correctly, It looks like the original request is redirected by the server. This leads to a new
Requestobject getting created and that's the one I eventually receive.This is happening at
HttpRedirector:312:I need some sort of event listener to let me know about the disposal of the original
Requestobject and the creation of the new one. Alternatively, I need a way of knowing that the first request is related to the last request. (The only piece of information I found to link the two isHttpRequest.getConversation()but the method isprotected). Comparing the URL, HTTP method is not enough because my client fires repeated requests on a regular basis.As far as I can tell, Jetty does not provide enough information to do either of these things :(
I am really hoping I don't have to disable redirects and handle them myself because my code is already very hairy doing all this connection tracking. I am trying desperately to hide these low-level details from my application code.
cowwoc commentedon Aug 2, 2018
I ended up using reflection to read
HttpRequest.getConversation(). This allowed me to mapResult.getRequest()back to the originalRequestobject. It's an ugly hack with poor performance, but it works :)Please let me know once you guys come up with something I can use long-term solution.
46 remaining items
github-actions commentedon Jun 17, 2026
This issue has been automatically marked as stale because it has been a
full year without activity. It will be closed if no further activity occurs.
Thank you for your contributions.
github-actions commentedon Jul 18, 2026
This issue has been closed due to it having no activity.