Skip to content

Support graceful shutdown of HTTP clientΒ #1445

Closed as not planned
Closed as not planned

Description

@sgrimm-sg

We use the Jetty HTTP client's async API and want it to shut down gracefully as part of our application's shutdown sequence, completing any in-progress requests.

Calling HttpClient.stop() aborts all the pending requests immediately, so we have to wait for requests to finish before calling it. There doesn't seem to be a documented API to query a client to see if it has any active requests. We've implemented a hacked-up solution by poking around the internal states of the HTTP client's various child objects. That approach is likely to break when we upgrade to new Jetty releases. A public API to either shut down gracefully or to query whether there are any outstanding requests would be far preferable.

Activity

  1. olamy commented on Apr 4, 2017

    @olamy
    Member

    You can use something such

    HttpDestination destination =
            (HttpDestination) httpClient.getDestinations().get( 0 );
        while (!destination.getHttpExchanges().isEmpty())
        {
          wait
        }
    

    Note you have to iterate on all destinations.
    @sbordet WDYT about adding a method for such use case? what about methods similar to ExecutorService such shutdownNow() and shutdown()?

  2. sgrimm-sg commented on Apr 4, 2017

    @sgrimm-sg
    Author

    Maybe a bit of a tangent, but checking for exchanges on the destination was one of the first things I tried and I couldn't get it to work. Here's a simple test case that fails for me.

      @Test
      public void testHttpExchangesWithOpenConnection() throws Exception {
        HttpClient client = new HttpClient();
        ServerSocket serverSocket = new ServerSocket(0);
    
        client.start();
        Request request = client.newRequest("http://localhost:" + serverSocket.getLocalPort());
        request.send(result -> System.out.println("Complete"));
    
        serverSocket.accept();
        Thread.sleep(500);  // test passes if this is commented out
    
        boolean hasAnyExchanges = client.getDestinations().stream()
            .map(dest -> ((HttpDestination) dest).getHttpExchanges())
            .anyMatch(exchanges -> !exchanges.isEmpty());
    
        // A connection is open and the server hasn't returned a response yet.
        assertTrue(hasAnyExchanges);
      }
    
  3. olamy commented on Apr 11, 2017

    @olamy
    Member

    actually it's fine :-) As you don't send back any content/response the client never go to response listener callback so you cannot see System.out.println("Complete")
    So here everything normal the http exchange has been finished..

  4. sgrimm-sg commented on Apr 11, 2017

    @sgrimm-sg
    Author

    If I were to shut down as soon as the exchange lists were all empty, I would be closing the connection during the middle of an active request. Maybe not a problem for a simple query or an idempotent operation, but imagine the request causes the server to take some real-world action like sending email. I'd never find out if the operation succeeded or returned an error of some kind. In an ideal world the server would give me a query API to find out the status of the operation after an unexpected connection close, but a lot of third-party services don't support that.

    My workaround is not wonderful, but gets the job done. In my initialization, I call httpClient.setRemoveIdleDestinations(true); and then in the application's shutdown code, after the parts of the application that can initiate new requests have been shut down:

      // The client is set to remove destinations that aren't in use, so the list of destinations
      // can be used as a proxy for the list of active connections. But if there are connection
      // errors, the client does not remove the destinations, so only wait for up to the maximum
      // request idle time. This may erroneously shut down a request that's trickling a response
      // back to us slowly.
      if (!httpClient.getDestinations().isEmpty()) {
        log.info("Waiting for outgoing HTTP requests to complete");
    
        long timeout = System.currentTimeMillis() + IDLE_TIME_MS;
        boolean foundConnection;
            
        do {
          Thread.sleep(100);
          foundConnection = httpClient.getDestinations().stream()
              .map(Destination::getHost)
              .anyMatch(host -> !"null".equals(host));  // yes, it uses the string "null"
        } while (foundConnection && System.currentTimeMillis() < timeout);
    
        if (foundConnection) {
          log.warn("Timed out waiting for outgoing HTTP requests to finish");
        } else {
          log.info("Outgoing HTTP requests finished");
        }
      }
    
      httpClient.stop();
    
  5. sbordet commented on Apr 11, 2017

    @sbordet
    Contributor

    I'm not sure I want to add this feature to HttpClient. On the other hand it can be easily implemented in application code.

    The idea is to guard sending of the requests until the application wants to shut it down. A boolean does the job, and when flipped it will just forbid further request sending.

    As for waiting that sent requests complete, this is easily achieved using a Phaser, by calling register() when a request is sent (easily done from a Request.QueuedListener), and by calling arrive() from a Response.CompleteListener.

    An alternative to Phaser could be an AtomicLong plus some waiting functionality.

    Once the Phaser completes, you can call HttpClient.stop().

  6. sgrimm-sg commented on Apr 12, 2017

    @sgrimm-sg
    Author

    For what it's worth, I initially considered something like what you suggest (keeping track of the number of requests) but wasn't crazy about the idea because it means that all the places in the application where HTTP requests are initiated have to make sure to add listeners to increment and decrement the count reliably; if a developer, say, adds a code path in a CompleteListener that can cause it to return without calling arrive(), the shutdown code will think the request is still in flight.

    That risk can be lowered by forbidding direct use of the request-sending API methods and writing some wrappers that add listeners, of course. But the HTTP client already knows exactly which requests are in what states and its list will always be 100% accurate even if the application code has bugs.

    Obviously, though, my perspective on what changes make sense to include in the library is pretty limited, so no problem if this isn't something worth considering.

  7. olamy commented on Nov 10, 2017

    @olamy
    Member

    IMHO can be close

  8. cowwoc commented on Jul 27, 2018

    @cowwoc
    Contributor

    I second @sgrimm-sg's point: implementing this in application code is not particularly trivial. Seeing as Jetty already has this information on-hand it would be great if it could use it (provide a graceful shutdown method) or expose it (so we can use it to implement graceful shutdown ourselves).

    In my own code, there is a strong separation between the factory that creates, starts, stops a HttpClient and code that uses it to initiate requests, parse responses. Ideally, I don't want to mix the two across they are logically far from each other (separation of concerns).

    @sbordet Regarding your suggestion, how would you handle new WebsocketClient(HttpClient)? I can't listen to Request.QueuedListener, Response.CompleteListener in that case.

  9. sbordet commented on Jul 29, 2018

    @sbordet
    Contributor

    There are a ton of issues to take care of to implement this feature in a generic way.

    Having the application know what they do will make much easier for applications to implement graceful shutdown.

    Issues to consider in case a "shutdown" is triggered:

    • What happens to requests sent after shutdown?
    • What happens to requests queued before shutdown?
    • What happens to conversations such as redirects or authentications if shutdown happens in the middle of the conversation?
    • What happens to idle requests/responses that are not complete? Wait for idle timeout or not?
    • What if your application logic implements a retry mechanism for failed requests?

    The notion of "graceful" is application dependent: some would like in-flight requests to finish, but won't wait for the idle timeout to expire, some don't care about redirects, but care about authentications, etc., etc.

    Making everybody happy will be very difficult, but we're open to suggestions for a truly generic solution.

  10. cowwoc commented on Jul 29, 2018

    @cowwoc
    Contributor

    @sbordet As an application developer, I think the following functionality would be useful (though obviously this list is not all-inclusive):

    • Ability to get the number of outstanding requests, and maybe even get the Request objects themselves (for troubleshooting purposes)
    • Ability to wait for the outstanding requests to hit 0, with a timeout. Let the application figure out what happens if a timeout occurs (that's out of scope for you).
    • Ability to forcibly close all outstanding requests.
    • Ability to mark the HttpClient in "shutdown in progress" mode. While it is in this mode, attempting to create new requests will throw an exception.
  11. sgrimm-sg commented on Jul 30, 2018

    @sgrimm-sg
    Author

    For my application, the big issue is whether or not a request has been sent to the remote host. If so, I need to wait for the request to finish or time out. If not, it is safe to immediately abort since the request can't have caused the remote host to perform any potentially non-idempotent actions.

    I agree with everything @cowwoc said above. To answer the specific questions (which I realize are not an exhaustive list):

    • Requests sent after shutdown: Throw an exception.
    • Requests queued before shutdown: Call their listeners with an exception indicating the requests were aborted due to shutdown.
    • Conversations such as redirects or authentications: Continue the requests until they complete. The library can't know if the original request that triggered the redirect or auth-needed response caused the server to take action. If the application cares about controlling this with finer granularity, it can tell the library to not automatically follow redirects, etc.
    • Idle requests/responses: Wait for them to finish or time out. The application can call the non-graceful shutdown method if it decides it doesn't want to wait any more.
    • Application retry logic: This isn't something I'd expect the HTTP client library to even attempt to address; it's up to the application to know if it's shutting down or not.

    In general I agree that it is impossible to come up with a single answer that will be completely correct for every application. But I think as long as the behavior is well-defined and supports waiting for in-progress things to finish, the exact definition of "in-progress" (which is what most of the questions are about) is less critical. For example, though it wouldn't be completely ideal for my application, I'd be satisfied if queued requests were processed rather than aborted or if requests could continue to be sent after shutdown.

  12. cowwoc commented on Jul 30, 2018

    @cowwoc
    Contributor

    @sgrimm-sg One could potentially allow queued requests to continue by default and have a separate method that explicitly cancels all queued requests. That should satisfy both possibilities.

  13. cowwoc commented on Aug 2, 2018

    @cowwoc
    Contributor

    @sbordet I tried implementing your suggestion, keeping track of connections using Request.QueuedListener, Response.CompleteListener (I'm actually using BufferingResponseListener.onComplete for the latter)

    It turns out that onComplete is invoked with a Result whose embedded Request object is not equal to the Request object that initiated the request (the URL and HTTP method are the same but the Request object is not). This means my code has no way of knowing that a previously-fired request has actually completed.

    Any suggestions?

  14. cowwoc commented on Aug 2, 2018

    @cowwoc
    Contributor

    I debugged this further. If I understand correctly, It looks like the original request is redirected by the server. This leads to a new Request object getting created and that's the one I eventually receive.

    This is happening at HttpRedirector:312:

    Request redirect = client.copyRequest(httpRequest, location);

    I need some sort of event listener to let me know about the disposal of the original Request object and the creation of the new one. Alternatively, I need a way of knowing that the first request is related to the last request. (The only piece of information I found to link the two is HttpRequest.getConversation() but the method is protected). Comparing the URL, HTTP method is not enough because my client fires repeated requests on a regular basis.

    As far as I can tell, Jetty does not provide enough information to do either of these things :(

    I am really hoping I don't have to disable redirects and handle them myself because my code is already very hairy doing all this connection tracking. I am trying desperately to hide these low-level details from my application code.

  15. cowwoc commented on Aug 2, 2018

    @cowwoc
    Contributor

    I ended up using reflection to read HttpRequest.getConversation(). This allowed me to map Result.getRequest() back to the original Request object. It's an ugly hack with poor performance, but it works :)

    Please let me know once you guys come up with something I can use long-term solution.

  16. 46 remaining items

  17. added this to Jetty 12.1.6 - FROZEN and removed this from Jetty 12.1.5on Dec 1, 2025
  18. github-actions commented on Jun 17, 2026

    @github-actions

    This issue has been automatically marked as stale because it has been a
    full year without activity. It will be closed if no further activity occurs.
    Thank you for your contributions.

  19. github-actions commented on Jul 18, 2026

    @github-actions

    This issue has been closed due to it having no activity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

StaleFor auto-closed stale issues and pull requests

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions