Skip to content

Next.js middleware tracing produces two traces per requestΒ #91282

Description

@blairmcalpine

Link to the code that reproduces this issue

https://github.com/blairmcalpine/double-trace-repro

To Reproduce

  1. npm i
  2. npm run dev (don't visit the site on your browser, it just makes a bunch of noisy traces)
  3. (Requires docker, feel free to use any other local opentelemetry collector you wish)
docker run --rm -it \
  -p 16686:16686 \
  -p 4317:4317 \
  jaegertracing/jaeger:latest
  1. curl http://localhost:3000/
  2. Visit localhost:16686. Filter by the double-trace-repro service, and observe that there is two entirely separate traces for the single request you sent to the server.

Current vs. Expected behavior

I expect that for one server request, there is one associated opentelemetry trace. This is however not the case, because middleware tracing emits its own, entirely separate trace that is not at all connected to the base one.

I can see this making sense previously when middleware ran in edge runtimes (so the work was done by an entirely separate machine), but now that we have proxy and the node runtime, it's always going to be handled by the same process. This currently happens because of how we invoke middleware, where we run an entirely new BaseServer.handleRequest span that gets its propagation info from the incoming requests' headers (ie not the propagation info from the already started parent span). Instead, tracing middleware should be tied to the parent GET /[template]/some-route span (that is also BaseServer.handleRequest).

This current behaviour is especially problematic if you want to use datadog for Next.js apps - this is because of datadog's trace metrics, which are computed from every single common trace emitted by a service. Because these two traces are both BaseServer.handleRequest, this means datadog things they are two separate requests. This means 2x the perceived traffic for a service, as well as incorrect averages for latency, etc.

To work around this I've had to create a "filter" span exporter that basically just drops this extra BaseServer.handleRequest span from middleware, but I'd love for it to just be included on the main trace instead like it should be.

Provide environment information

Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin Kernel Version 25.3.0: Wed Jan 28 20:48:41 PST 2026; root:xnu-12377.81.4~5/RELEASE_ARM64_T6041
  Available memory (MB): 49152
  Available CPU cores: 14
Binaries:
  Node: 24.13.0
  npm: 11.6.2
  Yarn: 1.22.22
  pnpm: 10.32.1
Relevant Packages:
  next: 16.1.6 // Latest available version is detected (16.1.6).
  eslint-config-next: N/A
  react: 19.2.3
  react-dom: 19.2.3
  typescript: 5.9.3
Next.js Config:
  output: N/A

Which area(s) are affected? (Select all that apply)

Instrumentation, Middleware

Which stage(s) are affected? (Select all that apply)

next dev (local), next start (local), Other (Deployed)

Additional context

No response

Activity

  1. added
    InstrumentationRelated to Next.js Instrumentation.
    MiddlewareRelated to Next.js Middleware.
    linear: nextConfirmed issue that is tracked by the Next.js team.
    on Mar 12, 2026
  2. khalidsaidi commented on Mar 25, 2026

    @khalidsaidi

    A2ABench has an accepted answer for this imported thread.

  3. arturkasperek-blip commented on Aug 31, 2026

    @arturkasperek-blip

    @marcoshernanz any chances to fix that problem?

  4. philwolstenholme commented on Sep 10, 2026

    @philwolstenholme
    Contributor

    It'd be great to have a review on #91419

  5. truesolarfire commented on Sep 18, 2026

    @truesolarfire

    There is already a working implementation of this: #91419. Worth reviving rather than rediscovering β€” it was not rejected. A maintainer requested review in July, no review landed, and the author closed his own PR in September after six months, noting he was no longer sure it was still relevant. Fixes #91282 was on it.

    Two things have changed since, both of which make the ask smaller.

    The fix is now a one-line condition

    #91419 targeted the shape it describes β€” two BaseServer.handleRequest calls each independently extracting from req.headers β€” and needed +60/βˆ’23 across three files including a new tracer.injectTraceContext() method.

    Current canary is explicit instead (next-server.ts:1822-1826):

    // Node.js middleware runs in-process, inside the active
    // `handleRequest` span. Detach that span so the middleware span
    // becomes a sibling root (or parents to an incoming traceparent),
    // matching edge middleware which runs in a detached sandbox.
    result = await getTracer().runWithDetachedContext(() => ...)

    with runWithDetachedContext being context.with(ROOT_CONTEXT, fn) (lib/trace/tracer.ts:286-291).

    The parity rationale is sound for edge, where the work genuinely happens in another sandbox. For in-process Node middleware it discards a parent that is right there. So the ask is not "add an injection API" any more, nor "remove the detach" β€” it is make that one ROOT_CONTEXT swap conditional on the runtime, and keep it for the edge path where it is needed.

    It also answers the objection raised on #91419

    @blairmcalpine's review point on that PR was that making middleware a child means the root span name becomes the APM resource name, and a root called GET rather than GET / would regress Datadog. That was fixed there by threading parsedUrl.pathname into the span name.

    Worth noting it does not arise at all if the context is published one layer up, at the http.Server level, because then the root is @opentelemetry/instrumentation-http's own SERVER span, which names itself from http.route. Next's span naming never becomes the resource name.

    A workaround that needs no Next change

    The parenthetical in that comment β€” "or parents to an incoming traceparent" β€” is usable today. Inside the detached context, withPropagatedContext finds no active span and falls through to propagation.extract over the request headers (tracer.ts:327-328). Publishing the active server span onto the incoming headers before Next reads them puts middleware back in the request's trace:

    // in an http.Server 'request' hook, inside the OTel server span
    if (!req.headers.traceparent && trace.getActiveSpan()) {
      propagation.inject(context.active(), req.headers)
    }

    Measured on a built standalone app, 3 requests, one of them an auth route whose middleware makes a real OIDC discovery call:

    spans traces
    before 11 6
    after 11 3

    This is worth preferring over the filter-exporter workaround in the issue body, because middleware's outbound calls are the part a filter cannot save. The discovery fetch is a child of the middleware span, so it lives in the orphan trace too: dropping the middleware span drops the auth dependency with it, and keeping it leaves that dependency in a trace unconnected to the request that caused it.

    Two caveats for anyone copying it. It leans on internal behaviour rather than public API, so it wants a test to guard upgrades. And the parent arrives marked remote, so ParentBasedSampler takes its remoteParentSampled / remoteParentNotSampled branches rather than the local ones β€” both inherit by default, so the effective behaviour is right.

  6. github-actions commented on Oct 15, 2026

    @github-actions
    Contributor

    This issue was closed by a maintainer. If you think it was closed by mistake, you can request that it be reopened within the next 14 days.

    Reply with Reopen: <reason> and explain why the issue is still relevant. If you opened this issue or commented before it was closed, the issue will be reopened automatically.

    After 14 days, please open a new issue with up-to-date details and a reproduction, when applicable.

  7. github-actions commented on Oct 29, 2026

    @github-actions
    Contributor

    This closed issue has been automatically locked because it had no new activity for 2 weeks. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.

  8. locked as resolved and limited conversation to collaborators on Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    InstrumentationRelated to Next.js Instrumentation.MiddlewareRelated to Next.js Middleware.linear: nextConfirmed issue that is tracked by the Next.js team.locked

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions