Repository navigation
[ESMFold2] Don't build the MSA encoder for checkpoints without one - #49890
Merged
Rocketknight1 merged 1 commit intoOct 27, 2026
Merged
Conversation
Contributor
|
Thank you for your contribution 🤗! CI Security Gate — waiting for maintainer approvalCI is approved automatically only for authors who already have a contribution merged in this repository. A maintainer will review this PR and approve CI to run on it. |
Rocketknight1
approved these changes
Oct 27, 2026
Rocketknight1
left a comment
Member
There was a problem hiding this comment.
Yes, this looks good! The port was written working mostly on the non-fast checkpoints, so this slipped through. Thank you!
Rocketknight1
force-pushed
the
esmfold2-msa-encoder-enabled
branch
from
October 27, 2026 12:14
8332c75 to
6f4d68a
Compare
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Contributor
|
[For maintainers] Suggested jobs to run (before merge) run-slow: esmfold2 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
EsmFold2Model.__init__always buildsself.msa_encoder, but checkpoints trained without MSA conditioning don't ship MSA-encoder weights.biohub/ESMFold2-Fast's config hasmsa_encoder.enabled: false, and its safetensors contain none of the 130msa_encoder.*tensors. The flag is parsed but never read, so loading the checkpoint leaves the MSA encoder randomly initialized. Becauseprepare_protein_featuresalways providesmsa(a single-row query MSA forinfer_protein),_build_msa_kwargsreturns MSA inputs, and the trunk runs the random encoder on every loop.The result is non-deterministic and wrong. With
biohub/ESMFold2-Faston CPU, two loads with different seeds (same sequence, same diffusion noise) give pLDDT 0.272 vs 0.287 and a 48.9 Å maximum coordinate difference. Passing no MSA makes the two loads identical. The referenceesmpackage builds the MSA encoder onlyif msa_cfg.enabled.The fix:
enabled(defaultTrue, sobiohub/ESMFold2and existing configs are unchanged) toEsmFold2MsaEncoderConfig.msa_encoderonly when it is enabled, otherwiseNone._build_msa_kwargsreturnNonewhen there is no MSA encoder, so MSA features are ignored for such checkpoints.modeling_esmfold2.pywas regenerated from the modular file.Tests
Adds
test_msa_encoder_disabled, which checks that withmsa_encoder={"enabled": False}there is no module and nomsa_encoder.*keys, that inference runs, and that the setting survivessave_pretrained/from_pretrained. The test fails onmainand passes with this change. Alltests/models/esmfold2non-slow tests pass on CPU (14 passed, 1 skipped).ruff check/ruff format --checkandutils/check_modular_conversion.pyare clean.Before submitting
Who can review?
@ArthurZucker @Cyrilvallez