Skip to content

fix(microservices): survive custom deserializer errors in clients - #17995

Merged
kamilmysliwiec merged 2 commits into
nestjs:masterfrom
johnnyhuirilef:fix/client-deserializer-errors
Oct 8, 2026
Merged

kamilmysliwiec merged 2 commits into
nestjs:masterfrom
johnnyhuirilef:fix/client-deserializer-errors

Conversation

@johnnyhuirilef

Copy link
Copy Markdown
Contributor

PR Checklist

PR Type

  • Bugfix

What is the current behavior?

Issue Number: #17994

Closes #17994

Hi! 👋 This is the follow-up to #17959, which fixed the same problem in ClientNats.

A custom deserializer that throws or rejects on a reply crashes the process. ClientTCP, ClientRedis, ClientMqtt and ClientRMQ await the deserializer inside a listener that nobody awaits. The rejection is unhandled, so Node exits. If the app handles it, the request never settles.

I ran the repro on 12.1.2 with real brokers. All four clients crashed with exit code 1 on that reply:

RESULT: BUG: 4 of 8 cases did not end as expected (a crashed client exits with code 1).

The default deserializer never throws, so only a custom deserializer triggers this.

What is the new behavior?

ClientRMQ catches the error and fails the exact request with it. The callback gets the error and isDisposed, as in ClientNats. The RMQ reply consumer also ignores the null message that RabbitMQ sends when it cancels the consumer, as ServerRMQ does. The client logs a warning in that case, because the pending requests can no longer get a reply.

ClientTCP, ClientRedis and ClientMqtt catch the error, log it and drop the reply. Only the deserializer knows the request id, so these clients cannot tell which request to fail. The other pending requests stay as they are.

The same repro against a local build of this branch:

RESULT: FIXED: no client crashed, and RMQ failed the request with the deserializer error.

The four valid replies still resolve with "ok".

Does this PR introduce a breaking change?

  • Yes
  • No

Valid replies take the same path as before. Nothing exported changes.

Other information

The specs cover these cases:

  • A deserializer that throws, and one that rejects.
  • ClientRMQ: the request fails with the error, and the client removes the request from routingMap.
  • ClientTCP, ClientRedis, ClientMqtt: the client logs the error and keeps the other pending requests.
  • An error from the request callback is not caught by the new try.
  • ClientRMQ: a null message from the broker does not throw, and the client logs a warning.
  • The specs check the exact log message.

Some limits stay:

  • On TCP, Redis and MQTT, the request that the rejected reply belonged to still waits, because the client cannot know its id. Nest sets no default request timeout. The RxJS timeout() operator on the caller side ends the wait and frees the request.
  • A deserializer that returns undefined instead of throwing still fails outside the new try.
  • ClientKafka has the same missing catch. kafkajs awaits the handler, logs the error and rethrows it to its own runner. I did not run that case, so I left it out of this PR.

I am happy to look at a better option for any of these if you have one in mind. 🙂

A custom deserializer that throws or rejects on a reply crashed the
process. The TCP, Redis, MQTT and RMQ clients await the deserializer
inside a listener that nobody awaits. The rejection was unhandled, so
Node exited. If the app handled it, the request never settled.

ClientRMQ now catches the error and fails the exact request with it.
The callback gets the error and isDisposed, like ClientNats does. The
RMQ reply consumer also skips the null message that RabbitMQ sends
when it cancels the consumer, as ServerRMQ does, and logs a warning.

ClientTCP, ClientRedis and ClientMqtt catch the error, log it and drop
the reply. Only the deserializer knows the request id, so these
clients cannot fail the request. Other pending requests stay untouched.

Closes nestjs#17994

@Asadshah7950 Asadshah7950 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pulled this down and verified the fix locally across all 4 microservice client suites (213 tests passing).

The crash it fixes is real. Because event callbacks like socket data listeners and message handlers run unawaited in the event loop, an unhandled rejection inside deserializer.deserialize() causes Node to exit with code 1.

A few things I like about this implementation:

  1. Transport-aware separation:

    • In ClientRMQ, correlation is tracked at the envelope level via msg.properties.correlationId. Because callback is already bound to the pending request, the catch block can cleanly fail that specific caller with { err: error, isDisposed: true } and clean up routingMap.
    • In ClientTCP, ClientRedis, and ClientMqtt, the correlation ID lives inside the payload itself. When the deserializer fails, the client can't know which request the message was meant for. Dropping the bad packet and logging with this.logger.error(...) prevents the crash without risking corrupted state in other pending requests.
  2. RabbitMQ consumer cancellation:

    • Adding if (!msg) in ClientRMQ.consumeChannel() handles the null delivery that RabbitMQ sends when a queue is deleted or cancelled, so we don't throw an uncaught TypeError on msg.properties.
  3. Narrow try/catch scope:

    • The try/catch specifically wraps this.deserializer.deserialize() and nothing downstream. If subscriber code inside callback() throws, it isn't swallowed by this catch block. I verified this with the callback failure tests.
  4. Non-blocking note on request cleanup for TCP/Redis/Mqtt:

    • For TCP, Redis, and MQTT, because dropped packets can't be mapped back to an ID, the caller's request will stay in routingMap until the caller's RxJS timeout() fires. That's the right tradeoff since guessing the ID would be dangerous.

Clean fix and thorough test coverage across all transports. LGTM!

The null-message guard left a cancelled reply consumer in place while
the client stayed connected, so replies stopped arriving for good. Without
it the error closes the connection and the reconnect sets the channel up
again, as before. Dropped responses are now logged with their stack.
@kamilmysliwiec
kamilmysliwiec merged commit d6d080d into nestjs:master Oct 8, 2026
4 checks passed
kamilmysliwiec added a commit that referenced this pull request Oct 8, 2026
…18043)

* fix(microservices): fail pending kafka request on deserializer error

a custom deserializer that throws on a response rejects the kafka reply handler.
kafkajs logs the error, commits the offset and rethrows, so the consumer crashes
and the pending request never resolves.

the other clients already catch this. mqtt, redis, tcp and rmq got the check in
PR #17995 and rmq fails the pending callback too, bcs the id is known before the
deserializer runs. kafka has the correlation id on the headers as well, so the
request can be failed instead of staying pending.

adds a focused spec in packages/microservices/test/client/client-kafka-deserializer.spec.ts,
the same shape as the sibling fixes from that sweep. it fails on master with
"promise rejected TypeError: Cannot read properties of undefined instead of resolving"

* fix(microservices): ignore kafka replies with an empty correlation id

---------

Co-authored-by: Kamil Myśliwiec <mail@kamilmysliwiec.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A custom deserializer error crashes the TCP, Redis, MQTT and RMQ clients

3 participants