Repository navigation
fix(core): roll back modules on failed lazy load - #18385
kamilmysliwiec merged 3 commits into
Conversation
When a provider of a lazily loaded module fails during instantiation or scanning encounters an error, load() previously left half-built modules registered in NestContainer. Subsequent load() calls returned the broken ModuleRef. Following up on nestjs#18371 (in-flight deduplication), safely roll back failed loads: - Track inserted modules during lazy scan and roll them back if scanning throws. - Roll back modulesToBuild if buildModules rejects. - Use identity check in NestContainer.removeModules to avoid clobbering concurrent or retried instances. - Remove provider and controller metadata entries in DiscoverableMetaHostCollection.
agammann
left a comment
There was a problem hiding this comment.
A failed dynamic import can leave the parent module in the container, so this rollback does not make the next lazy load retryable. setModule() calls this.modules.set(token, moduleRef) and then awaits addDynamicMetadata() (which recursively calls addDynamicModules()); addModule() pushes moduleRef into insertedModules only after that await returns. If an imported dynamic module fails to compile/insert, the parent was inserted but never added to the rollback list, so scanForModules() catches the error and removes only the children. A subsequent load() sees the leftover parent as already present. Please register the parent for rollback immediately after this.modules.set, or remove it in setModule when dynamic metadata insertion rejects; a failing dynamic imports case would cover this path.
kamilmysliwiec
left a comment
There was a problem hiding this comment.
Thanks for picking this up. A few things need to change before this can be reviewed in depth:
- It doesn't build.
packages/core/scanner.ts:617:TS2554: Expected 3 arguments, but got 4.insertOrOverrideModule()passesinsertedModulestooverrideModule(), which still takes three. - Overridden modules aren't tracked.
NestContainer#replaceModule()gains aninsertedModulesparameter but never uses it: it doesn't pass it tosetModule()and doesn't record the module. A module replaced throughoverrideModule()(e.g. in a testing module) that fails to load is therefore left registered, which is the case this PR sets out to fix. - Global modules bound in the meantime. Removing a failed
@Global()module fromglobalModulesdoesn't undobindGlobalsToImports(). A module that anotherload()scanned while this one was in flight keeps the removed module in its imports. Rolling back has to unlink it from every module it was bound to, or the global modules have to be bound only once the load succeeded. - Unrelated changes. The
NestContainerconstructor is reformatted (likely a different Prettier version); please run the repository'snpm run formatso the diff only holds the fix.
Once it builds, the tests also need to cover points 2 and 3: a failed load of an overridden module, and a failed global module next to a concurrent load of another module.
) - Fix scanner.overrideModule signature and forward insertedModules to replaceModule. - Track inserted module in setModule immediately and clean up on dynamic metadata error. - Unlink rolled-back modules from module.imports across remaining modules. - Revert unrelated constructor formatting diff in NestContainer. - Add unit tests covering overridden modules, concurrent globals, and dynamic import failures.
|
Hi @kamilmysliwiec, @agammann, Thank you both for the detailed and constructive reviews! I've addressed all points in the latest commit:
|
kamilmysliwiec
left a comment
There was a problem hiding this comment.
Thanks, this covers everything from the last round. Two small things are left:
-
A slow sibling can outlive the rollback.
addDynamicModules()still usesPromise.all(). If a dynamic module'simportshold one import that rejects and one that is still pending (a slowimport(), say), the load rolls back as soon as the first one rejects. The pending one then reachessetModule()and stays in the container, because it's pushed toinsertedModulesafterremoveModules()has already read the list.Please wait for all of them with
Promise.allSettled()and rethrow the first rejection, so that every sibling is tracked before the rollback. A test for it: a dynamic module withimports: [Promise.reject(err), slowPromise], expecting an empty container once the slow import has resolved. -
The
try/catchinsetModule()is no longer needed. The parent is now ininsertedModulesbefore theawait, soremoveModules()already removes it. With thecatchin place,removeModules()skips that module instead, because its token no longer maps to it. Dropping thecatchalso leaves the bootstrap path as it is on master.
…setModule try-catch (nestjs#18385) - Use Promise.allSettled() in addDynamicModules() so pending siblings are tracked before throwing rejection. - Drop try/catch in setModule() since parent is registered in insertedModules and cleanly removed by removeModules(). - Add unit test for dynamic imports with slow sibling.
|
Hi @kamilmysliwiec, Thanks for the great feedback! Both points have been updated in the latest commit:
|
PR Checklist
Please check if your PR fulfills the following requirements:
PR Type
What kind of change does this PR introduce?
What is the current behavior?
When a provider of a lazily loaded module throws during
load()(or when scanning encounters an invalid/undefined import),LazyModuleLoader.load()rejects, but leaves the partially constructed module registered inNestContainer.Because
createPrototypes()marks the module class instantiated before resolving providers,Module#isInstantiatedis alreadytrue. On any subsequentload(),scanForModules()seeslazy && !moduleInserted && moduleInstance?.isInstantiated, returns[], andload()hands back the brokenModuleRefwhose failed providers holdnull/ uninstantiated state. The module never heals.If the failing module is
@Global(), it also permanently remains inglobalModules, causing subsequent lazy modules to bind to the dead global instance.In #18183, @kamilmysliwiec noted that a safe rollback required two things:
load()calls of the same module sharing a single in-flight load (which was merged in fix(core): share one in-flight load between concurrent lazy loads #18371).removeModules()so late rollbacks never delete modules rebuilt by retries or owned by concurrent successful loads.Issue Number: N/A
What is the new behavior?
LazyModuleLoader.load(), scanning is wrapped in atry/catch. Newly inserted modules are tracked during the scan; if scanning throws midway (e.g. an undefined or invalid nested import), any modules inserted during that scan are cleanly removed viaDependenciesScanner.removeModules().buildModulesfails duringcreateInstancesOfDependencies()or dependency scanning,modulesToBuild(the modules this load was responsible for building) are removed viaDependenciesScanner.removeModules(modulesToBuild).NestContainer.removeModules(), an identity checkif (this.modules.get(moduleRef.token) !== moduleRef) continue;ensures we only delete if the module stored in the container is the exact instance being rolled back. This prevents late rollbacks from deleting a retry that already succeeded.DiscoverableMetaHostCollection.removeInstanceWrappers()cleans up discovery entries for both providers and controllers by instance wrapper ID.AModuleandBModuleboth importingSharedModulewhereAfails andBsucceeds) do not evict modules owned by another load.Does this PR introduce a breaking change?
The public API only adds
NestContainer.removeModules(),DependenciesScanner.removeModules(), andDiscoverableMetaHostCollection.removeInstanceWrappers().Other information
load()after first failed.NestContainer.removeModulesclears global, dynamic, provider discovery, and controller discovery entries, and ignores mismatched tokens.vitest run packages/core/test/injector,vitest run packages/core/test/discovery,vitest run --config vitest.config.integration.mts integration/lazy-modules).