Repository navigation
[improve][misc] PIP-472: Drop Jetty ee8 usage and adapt legacy javax.servlet additional servlets - #26444
Conversation
…servlet additional servlets PIP-472 kept a second Jetty environment (ee8) alive alongside ee10 so that AdditionalServlet plugins written against javax.servlet would keep working without recompilation. Each Jetty environment owns its own servlet context, servlet handler and filter chain, and Pulsar's broker/proxy filters are jakarta.servlet-typed, so servlets registered in the ee8 environment ran outside the request pipeline every other servlet goes through: the requiresAuthentication argument handed to addServletEe8 had no effect. Drop Jetty's ee8 API from Pulsar's web tier and adapt a legacy javax.servlet handler to jakarta.servlet with the Apache Felix org.apache.felix.http.jakartawrappers.ServletWrapper, registering it through the ordinary ee10 path. Both flavours now share one environment and one filter chain. Assisted-by: Claude Code (Opus 5)
david-streamlio
left a comment
There was a problem hiding this comment.
Reviewed at 0443f23. I verified the substantive claims independently rather than reading the diff
and agreeing with it — the design argument holds up and the tests earn their place.
Reproduced the motivation. I put BrokerAdditionalServletFilterChainTest and
LegacyJavaxAdditionalServlet onto the pre-change tree (36ff86c) unmodified and ran them:
testJavaxAdditionalServletGoesThroughTheFilterChain fails with
expected: 401 but was: 200, and passes on this branch. So the test is specific to the environment
split rather than to anything else the PR touches, exactly as you describe. Both new filter-chain
tests pass here (BrokerAdditionalServletFilterChainTest 2/2, ProxyAdditionalServletFilterChainTest
1/1), and :pulsar-broker-common:compileJava, :pulsar-broker:compileTestJava,
:pulsar-proxy:compileTestJava are clean.
Checked the dependency claims. org.apache.felix.http.wrappers:1.1.10 is 93,676 bytes, and its
POM declares only javax.servlet-api:4.0.1 and jakarta.servlet-api:6.0.0 (both provided, so
nothing transitive) plus provided/test annotations and test libs — matching the description. It is
compiled against exactly the 4.0.1 the catalog pins. The wrapper set is complete for the paths an
additional servlet can take: AsyncContextWrapper, AsyncListenerWrapper, PartWrapper,
PushBuilderWrapper, ReadListenerWrapper, ServletInputStream/OutputStreamWrapper,
RequestDispatcherWrapper, HttpSessionWrapper. That was my main worry with an adapter approach and
it looks well covered.
Checked the cleanup is complete.
- No
addServletEe8and noorg.eclipse.jetty.ee8reference anywhere in the tree after this PR
exceptjetty-upgrade/, and that module genuinely useslibs.jetty.ee8.servlet
(PrometheusMetricsProvider,JettyAdminServer), so keeping that one alias is right. - The three dropped aliases (
jetty-ee8-servlets,jetty-ee8-proxy,
jetty-ee8-websocket-jetty-server) are referenced by zero build files on master. PulsarService.addBrokerAdditionalServletsandProxyServiceStarter.addWebServerHandlersare the
only two consumers ofAdditionalServletsoutside the SPI itself, so converting both covers every
registration path.- Every module whose sources mention
javax.servletnow declares the dependency itself
(pulsar-broker-commonimplementation,pulsar-broker/pulsar-proxytestImplementation,
java-test-pluginscompileOnly) — nothing was relying on theapi(libs.jetty.ee8.servlet)that
this PR removes.
toJakartaServlet also quietly fixes a latent NPE: the old branches called
servletInstance.getClass().getName() in the mismatch log, which would itself throw if
getServletInstance() returned null. The new message handles null.
Two nits, neither blocking:
- Both filter-chain tests build a
new OkHttpClient()per request and never shut down its
dispatcher or connection pool. One client as a field, shut down in the@AfterClass, would keep
the thread pools bounded — it is the kind of thing that only shows up as flakiness under a loaded
CI runner. ProxyAdditionalServletFilterChainTestcovers only thejavaxcase, while the broker test covers
javaxandjakarta. Thejakartahalf is what shows the assertion is specific to the legacy
path rather than to proxy filter wiring in general, so it is worth the few lines on the proxy side
too.
…xy jakarta case - Both filter-chain tests now build a single OkHttpClient in setup and shut down its dispatcher and connection pool in cleanup, instead of one client per request. - ProxyAdditionalServletFilterChainTest gains the jakarta.servlet case, so the proxy assertion is shown to be specific to the legacy path rather than to proxy filter wiring in general. The jakarta fixture moves out of the broker test into a shared JakartaAdditionalServlet that mirrors LegacyJavaxAdditionalServlet, so both flavours echo the same request parts and can be compared directly. Assisted-by: Claude Code (Opus 5)
|
Thanks for the careful review — and for reproducing the motivation on the pre-change tree rather than taking the description on faith. Both nits addressed in 30d7fe2:
Verified locally: On the previous run, the only red job was |
|
CI is green — all 44 checks pass on 30d7fe2. The one red job on the first pass of this commit was @david-streamlio re-requested — the two nits are addressed in 30d7fe2. |
…servlet additional servlets (apache#26444)
PIP: PIP-472 (#25700), follow-up to #25912
Motivation
PIP-472 (#25912) moved Pulsar's web tier from Jetty's
ee8environment (javax.servlet) toee10(
jakarta.servlet). To keepAdditionalServletplugins that are written againstjavax.servletworking without recompilation, it kept a second Jetty environment —
ee8— alive inside thesame Jetty
Serverand registered those plugins there, viaWebService.addServletEe8(broker) andWebServer.addServletEe8(proxy).Combining both environments in one Jetty instance does not work the way that design assumed. A
Jetty environment owns its own servlet context, servlet handler and — crucially — its own filter
chain; the two are not shared. Pulsar's broker and proxy filters are
jakarta.servlet-typed andcan therefore only be installed on
ee10contexts, so a servlet registered in theee8environmentran outside the request pipeline that every other servlet goes through. Concretely, an
AdditionalServletreportingAdditionalServletType.JAVAX_SERVLETwas served without:RateLimitingFilterAuthenticationFilter(so therequiresAuthentication/isAuthenticationEnabledargument passed toaddServletEe8had no effect)WaitUntilPulsarServiceIsReadyForIncomingRequestsFilter,AddListenerAttributeFilterPreInterceptFilter/ProcessHandlerFilter/ResponseHandlerFilter(broker interceptors)DisableDebugHttpMethodFilter,MaxRequestSizeFilterCustomHeaderFilter(proxyHttpResponseHeadersJson)setDecodeAmbiguousURIs(true)popularServletParams(...)(request buffer size)Keeping the two environments would mean maintaining two parallel, differently-typed filter chains
for the rest of PIP-472's lifetime. Running one environment and adapting the legacy servlets into it
is simpler and gives every additional servlet the same request pipeline.
Modifications
ee8API from Pulsar's web tier.WebService.addServletEe8andWebServer.addServletEe8are deleted (both were introduced by [improve][misc] PIP-472: Migrate from javax.* to jakarta.* APIs #25912 and have never been in arelease), and
api(libs.jetty.ee8.servlet)is removed frompulsar-brokerandpulsar-proxy.javax.servletadditional servlets instead. A newAdditionalServletUtils.toJakartaServlet(AdditionalServlet)inpulsar-broker-commonreturns aJAKARTA_SERVLETinstance as it is, and wraps aJAVAX_SERVLETinstance inorg.apache.felix.http.jakartawrappers.ServletWrapperfromorg.apache.felix:org.apache.felix.http.wrappers:1.1.10(Apache-2.0; ~91 KB; its only dependenciesare the two servlet APIs, both already on the classpath). Both flavours are then registered through
the ordinary
addServletpath.PulsarService.addBrokerAdditionalServletsand
ProxyServiceStarter.addWebServerHandlers: theJAVAX_SERVLET/JAKARTA_SERVLETswitch andits duplicated
instanceof-check-log-close-skip blocks become one call. Behaviour on a mismatchedservlet instance is unchanged (log an error, close the servlet, skip it).
felix-http-wrappers; drop thejetty-ee8-servlets,jetty-ee8-proxyand
jetty-ee8-websocket-jetty-serveraliases, which no build file referenced.jetty-ee8-servletstays for
jetty-upgrade/'s patched ZooKeeper admin server, which implements ZooKeeper's ownjavax.servlet-based APIs and runs a separate Jetty server.javax.servlet:javax.servlet-api:4.0.1stays as the API of legacy plugins (4.0 is a floor, not a preference: the Felix wrappers reference
Servlet 4.0 types such as
javax.servlet.http.PushBuilder), and moves topulsar-broker-commonalongside the Felix dependency.
ee8leftovers:MockBrokerServicemoves fromorg.eclipse.jetty.ee8.nested.*to anee10ServletContextHandlerwith ajakarta.servletHttpServlet, andPrometheusMetricsTestUtilnow importsorg.eclipse.jetty.ee10.servlet.HttpOutput, matchingPulsarPrometheusMetricsServlet. After this PR no Pulsar source outsidejetty-upgrade/referencesa Jetty
ee8class.pip/pip-472.mdis amended in place to describe the single-environment design, why theee8/ee10coexistence was replaced, and the added/removed dependencies.Verifying this change
This change added tests and can be verified as follows:
BrokerAdditionalServletFilterChainTestandProxyAdditionalServletFilterChainTestregister ajavax.servletadditional servlet with authentication enabled and assert that an unauthenticatedrequest is rejected with
401while an authenticated one gets200and the expected body. Bothfail on the pre-change code (verified locally:
expected: 401 but was: 200for thejavaxcase;the
jakartacase passes on both sides, so the test is specific to the environment split).BrokerAdditionalServletTestandProxyAdditionalServletTestgained a legacyjavax.servletadditional servlet next to the existing
jakarta.servletone. The sharedLegacyJavaxAdditionalServletfixture echoes the query parameter, request URI and method itobserves through the
javaxAPI, so a mistranslated request is visible; the proxy test also assertsthe
CustomHeaderFilterresponse header now reaches thejavaxservlet.TestAdditionalServletscovers the real NAR path —RandomAdditionalServletis a
javax.servletAdditionalServletloaded throughNarClassLoaderfrom/pulsar/examples.Ran locally against a freshly built image:
testBrokerAdditionalServletandtestProxyAdditionalServletboth pass../gradlew quickCheck,./gradlew sanityCheck,./gradlew :distribution:pulsar-server-distribution:checkBinaryLicenseand
./gradlew :distribution:pulsar-shell-distribution:checkBinaryLicenseall pass. Thejetty-ee8-*and
jetty-servlet-apientries stay inLICENSE.bin.txtbecause the patched ZooKeeper admin serverstill brings them into
lib/.Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
Adds
org.apache.felix:org.apache.felix.http.wrappers:1.1.10(Apache-2.0) topulsar-broker-commonand removes
org.eclipse.jetty.ee8:jetty-ee8-servletfrompulsar-brokerandpulsar-proxy.The
AdditionalServletSPI itself is unchanged: plugins reportingJAVAX_SERVLETand pluginsreporting
JAKARTA_SERVLETboth keep working, without recompilation.addServletEe8is removed, butit only ever existed on master (5.0.0-M1-SNAPSHOT) and is not part of any release.