Skip to content

File name lookup issue when using --venv-repositoryΒ #2926

Closed

Description

@Skunnyk

Hi, and thank you for making pex!

I'm try to use the new --venv-repository option from pex 2.59.

I have some strange issues (race condition I guess), when using --venv-repository, --layout packed and manylinux wheels.

How to reproduce (pex 2.59.2/ python 3.13, but also on 3.11):

skunnyk@dellboxe:/tmp $ rm -rf ~/.cache/pex/
skunnyk@dellboxe:/tmp $ python -m venv myvenv
skunnyk@dellboxe:/tmp $ source myvenv/bin/activate
(myvenv) skunnyk@dellboxe:/tmp $ pip install tokenizers==0.21.0 pex
Collecting tokenizers==0.21.0
  Using cached tokenizers-0.21.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB)
Collecting pex
  Using cached pex-2.59.2-py2.py3-none-any.whl.metadata (7.4 kB)
Collecting huggingface-hub<1.0,>=0.16.4 (from tokenizers==0.21.0)
  Using cached huggingface_hub-0.35.3-py3-none-any.whl.metadata (14 kB)
Collecting filelock (from huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
  Using cached filelock-3.19.1-py3-none-any.whl.metadata (2.1 kB)
Collecting fsspec>=2023.5.0 (from huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
  Using cached fsspec-2025.9.0-py3-none-any.whl.metadata (10 kB)
Collecting packaging>=20.9 (from huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
  Using cached packaging-25.0-py3-none-any.whl.metadata (3.3 kB)
Collecting pyyaml>=5.1 (from huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
  Using cached pyyaml-6.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (2.4 kB)
Collecting requests (from huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
  Using cached requests-2.32.5-py3-none-any.whl.metadata (4.9 kB)
Collecting tqdm>=4.42.1 (from huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
  Using cached tqdm-4.67.1-py3-none-any.whl.metadata (57 kB)
Collecting typing-extensions>=3.7.4.3 (from huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
  Using cached typing_extensions-4.15.0-py3-none-any.whl.metadata (3.3 kB)
Collecting hf-xet<2.0.0,>=1.1.3 (from huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
  Using cached hf_xet-1.1.10-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.7 kB)
Collecting charset_normalizer<4,>=2 (from requests->huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
  Using cached charset_normalizer-3.4.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (36 kB)
Collecting idna<4,>=2.5 (from requests->huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
  Using cached idna-3.10-py3-none-any.whl.metadata (10 kB)
Collecting urllib3<3,>=1.21.1 (from requests->huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
  Using cached urllib3-2.5.0-py3-none-any.whl.metadata (6.5 kB)
Collecting certifi>=2017.4.17 (from requests->huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
  Using cached certifi-2025.10.5-py3-none-any.whl.metadata (2.5 kB)
Using cached tokenizers-0.21.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached huggingface_hub-0.35.3-py3-none-any.whl (564 kB)
Using cached hf_xet-1.1.10-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.2 MB)
Using cached pex-2.59.2-py2.py3-none-any.whl (3.9 MB)
Using cached fsspec-2025.9.0-py3-none-any.whl (199 kB)
Using cached packaging-25.0-py3-none-any.whl (66 kB)
Using cached pyyaml-6.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (801 kB)
Using cached tqdm-4.67.1-py3-none-any.whl (78 kB)
Using cached typing_extensions-4.15.0-py3-none-any.whl (44 kB)
Using cached filelock-3.19.1-py3-none-any.whl (15 kB)
Using cached requests-2.32.5-py3-none-any.whl (64 kB)
Using cached charset_normalizer-3.4.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (151 kB)
Using cached idna-3.10-py3-none-any.whl (70 kB)
Using cached urllib3-2.5.0-py3-none-any.whl (129 kB)
Using cached certifi-2025.10.5-py3-none-any.whl (163 kB)
Installing collected packages: urllib3, typing-extensions, tqdm, pyyaml, pex, packaging, idna, hf-xet, fsspec, filelock, charset_normalizer, certifi, requests, huggingface-hub, tokenizers
Successfully installed certifi-2025.10.5 charset_normalizer-3.4.3 filelock-3.19.1 fsspec-2025.9.0 hf-xet-1.1.10 huggingface-hub-0.35.3 idna-3.10 packaging-25.0 pex-2.59.2 pyyaml-6.0.3 requests-2.32.5 tokenizers-0.21.0 tqdm-4.67.1 typing-extensions-4.15.0 urllib3-2.5.0
(myvenv) skunnyk@dellboxe:/tmp $ pex --venv-repository myvenv/ --layout packed  -o mypex

Here we are ok, I have a packed PEX in mypex.
If I rerun the command, errros happens:

(myvenv) skunnyk@dellboxe:/tmp $ pex --venv-repository myvenv/ --layout packed  -o mypex
[Errno 2] No such file or directory: '/home/skunnyk/.cache/pex/packed_wheels/0/14d5b2d393a9562265232fce7979b90f65992e51a5e77a697e9412209ba74d4e/hf-xet-1.1.10-cp37-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl' -> 'mypex~/.deps/hf-xet-1.1.10-cp37-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl'
(myvenv) skunnyk@dellboxe:/tmp $ pex --venv-repository myvenv/ --layout packed  -o mypex
Previous binary unexpectedly exists, cleaning: mypex
[Errno 2] No such file or directory: '/home/skunnyk/.cache/pex/packed_wheels/0/14d5b2d393a9562265232fce7979b90f65992e51a5e77a697e9412209ba74d4e/hf-xet-1.1.10-cp37-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl' -> 'mypex~/.deps/hf-xet-1.1.10-cp37-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl'
(myvenv) skunnyk@dellboxe:/tmp $ pex --venv-repository myvenv/ --layout packed  -o mypex
Previous binary unexpectedly exists, cleaning: mypex

After 4 or 5 attempts, it… works.

The strange thing is that it tries to lookup hf-xet-1.1.10-cp37-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl, while hf-xet-1.1.10-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl is present in the cache! (please note the switched manylinux2014 vs manylinux_2_17 in the file name !)

(myvenv) skunnyk@dellboxe:/tmp $ ls -l /home/skunnyk/.cache/pex/packed_wheels/0/14d5b2d393a9562265232fce7979b90f65992e51a5e77a697e9412209ba74d4e/
total 3120
-rw-r--r-- 1 skunnyk skunnyk 3187271 Oct  7 16:23 hf-xet-1.1.10-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

I isolated the problem to have a simple reproducer, but it seems that it happens when using manylinux_2_17_x86_64.manylinux2014 wheels (with the 2 manylinux names in wheel name).

Thank you,

Activity

  1. self-assigned this
    on Oct 7, 2025
  2. jsirois commented on Oct 7, 2025

    @jsirois
    Member

    Thanks @Skunnyk - I repro. This does in fact have something to do with wheels with compressed tag sets as you noticed. Somehow the order of the tags in the compressed set is unstable in some cases.

    I haven't sussed the issue yet, but should be able to get a fix released today - repros are gold; thanks again.

    I am curious why you used --layout packed. That's fairly niche and I'm always curious to know what People are doing with Pex. Can you share why you use that option and how you stumbled upon it being beneficial for your use case?

  3. jsirois commented on Oct 7, 2025

    @jsirois
    Member

    Aha, it looks like maturin (of a certain age at least) produces bogus WHEEL metadata: https://packaging.python.org/en/latest/specifications/binary-distribution-format/#file-contents

    :; unzip -qc /home/jsirois/.cache/pex/packed_wheels/0/71a8c97af7ce119371edccefee9413d263d90340d4b2905dfb8a04c5cdf6f840/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl tokenizers-0.21.0.dist-info/WHEEL
    Wheel-Version: 1.0
    Generator: maturin (1.7.5)
    Root-Is-Purelib: false
    Tag: cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64

    That should be like so with the compressed tag expanded:

    Wheel-Version: 1.0
    Generator: maturin (1.7.5)
    Root-Is-Purelib: false
    Tag: cp39-abi3-manylinux_2_17_x86_64
    Tag: cp39-abi3-manylinux2014_x86_64
    

    I'm not sure this is a cause, but note to self anyhow.

  4. jsirois commented on Oct 7, 2025

    @jsirois
    Member

    And lo and behold:

    ; unzip -qc /home/jsirois/.cache/pex/packed_wheels/0/14d5b2d393a9562265232fce7979b90f65992e51a5e77a697e9412209ba74d4e/hf-xet-1.1.10-cp37-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl hf_xet-1.1.10.dist-info/WHEEL
    Wheel-Version: 1.0
    Generator: maturin (1.9.4)
    Root-Is-Purelib: false
    Tag: cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64

    Same issue, same build backend (though newer maturin version this time).

  5. Skunnyk commented on Oct 7, 2025

    @Skunnyk
    Author

    Hi,

    Thank you! 🩡
    We are using --layout packed because the generated PEXes are >4GB on some of our apps (you know, pytorch, cuda and ML stuffs… ).

  6. jsirois commented on Oct 7, 2025

    @jsirois
    Member

    We are using --layout packed because the generated PEXes are >4GB on some of our apps (you know, pytorch, cuda and ML stuffs… ).

    Ok - makes sense. Are they actually >4GB and not just >2GB,<4GB? There are a few things to note here:

    1. The true 4GB limitation is gone in Python 3.14 3.13 (See: https://docs.python.org/3/library/zipimport.html namely Changed in version 3.13: ZIP64 is supported). Pex has tests for this:
      @pytest.mark.skipif(
      sys.version_info == (3, 13, 0, "alpha", 6),
      reason=(
      "The 3.13.0a6 release has a bug in its zipimporter ZIP64 handling, see: "
      "https://github.com/python/cpython/issues/118107"
      ),
      )
      @pytest.mark.skipif(
      subprocess.call(args=["cat", os.devnull]) != 0,
      reason="The cat binary is required for this test.",
      )
      def test_check(tmpdir):
      # type: (Any) -> None
      def assert_perform_check_zip64_handling(
      zipapp, # type: str
      test_run=True, # type: bool
      ):
      # type: (...) -> None
      assert Check.NONE.perform_check(Layout.ZIPAPP, zipapp) is None
      assert Check.ERROR.perform_check(Layout.PACKED, zipapp) is None
      assert Check.ERROR.perform_check(Layout.LOOSE, zipapp) is None
      # Python 3.13 newly supports ZIP64 in `zipimport.zipimporter`.
      if sys.version_info >= (3, 13):
      assert Check.WARN.perform_check(Layout.ZIPAPP, zipapp) is True
      assert Check.ERROR.perform_check(Layout.ZIPAPP, zipapp) is True
      if test_run:
      subprocess.check_call(args=[sys.executable, zipapp])
      else:
      expected_error_message_re = r"The PEX zip at {path} is not a valid zipapp: ".format(
      path=re.escape(zipapp)
      )
      with pytest.warns(PEXWarning, match=expected_error_message_re):
      assert Check.WARN.perform_check(Layout.ZIPAPP, zipapp) is False
      with pytest.raises(InvalidZipAppError, match=expected_error_message_re):
      Check.ERROR.perform_check(Layout.ZIPAPP, zipapp)
      if test_run:
      assert subprocess.call(args=[sys.executable, zipapp]) != 0
      @contextmanager
      def write_zipapp(path):
      # type: (str) -> Iterator[ZipFile]
      with open_zip(path, "w") as zip_file:
      yield zip_file
      zip_file.writestr("__main__.py", "print('BOOTED')")
      # N.B.: This test creates very large files in /tmp; so to relieve possible memory pressure in
      # tmpfs-based /tmp filesystems, we unlink files as soon as we don't need them instead of waiting
      # for tmpdir fixture cleanup at the end of the test.
      file_too_big = os.path.join(str(tmpdir), "file_too_big.py")
      with open(file_too_big, "w") as fp:
      fp.write("\n")
      accum = file_too_big + ".accum"
      for _ in range(32):
      safe_rename(file_too_big, accum)
      with open(file_too_big, "wb") as dest:
      subprocess.check_call(args=["cat", accum, accum], stdout=dest.fileno())
      os.unlink(accum)
      assert os.path.getsize(file_too_big) > 0xFFFFFFFF, (
      "ZIP64 must be used if file sizes are bigger than 0xFFFFFFFF per "
      "https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT 4.3.9.2"
      )
      zipapp_too_big = os.path.join(str(tmpdir), "too-big.pyz")
      with write_zipapp(zipapp_too_big) as zf:
      zf.write(file_too_big, os.path.basename(file_too_big))
      os.unlink(file_too_big)
      assert_perform_check_zip64_handling(
      zipapp_too_big,
      # N.B.: PyPy < 3.8 hangs when trying to run a zip64 app with a too-large file entry.
      test_run=not IS_PYPY or sys.version_info[:2] >= (3, 8),
      )
      os.unlink(zipapp_too_big)
      zipapp_too_many_entries = os.path.join(str(tmpdir), "too-many-entries.pyz")
      with write_zipapp(zipapp_too_many_entries) as zf:
      # N.B.: ZIP64 must be used if the number of central directory records is greater than
      # 0xFFFF per https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT 4.4.21
      for x in range(0xFFFF):
      zf.writestr(
      "module{x}.py".format(x=x),
      "# This is python module number {x}.".format(x=x).encode("utf-8"),
      )
      assert_perform_check_zip64_handling(zipapp_too_many_entries)
      zipapp_ok = os.path.join(str(tmpdir), "ok.pyz")
      with write_zipapp(zipapp_ok):
      pass
      for layout in Layout.values():
      for check in Check.values():
      assert check.perform_check(layout, zipapp_ok) is (
      True if layout is Layout.ZIPAPP and check is not Check.NONE else None
      )
      assert b"BOOTED\n" == subprocess.check_output(args=[sys.executable, zipapp_ok])
    2. The fake 2GB limitation caused by a sordid history in zipfile has been monkeypatched by Pex since the 2.33.3 release and is truly 4GB. (See: Enable true Zip64 support.Β #2714)
  7. Skunnyk commented on Oct 7, 2025

    @Skunnyk
    Author

    Yai, 1GB,2GB, 4GB+, we are using --layout packed everywhere to have consistent build options.

    Noted for python 3.13, will try it when time comes! (we are still on python 3.11)

  8. jsirois commented on Oct 7, 2025

    @jsirois
    Member

    Ok, yeah - looking deeper: The only place Pex uses WHEEL Tag metadata is to reconstruct the wheel file name from an an installed wheel in a venv (the case here of course!). It uses packaging.tags.parse_tag to get this metadata:

    pex/pex/wheel.py

    Lines 85 to 92 in 64c869c

    @property
    def tags(self):
    # type: () -> Tuple[tags.Tag, ...]
    return tuple(
    itertools.chain.from_iterable(
    tags.parse_tag(tag) for tag in self.metadata.get_all("Tag", ())
    )
    )

    That code, in turn does the . splitting (compressed tag expansion) and collects the tags in a frozenset. As a result, we see the flaky behavior here.

    Still looking at a workaround...

    After finding one, I'll get an issue filed with maturin. I checked and their code still emits bad metadata like this; so this is an ongoing problem.

  9. added a commit that references this issue on Oct 7, 2025
    62cbc09
  10. jsirois commented on Oct 7, 2025

    @jsirois
    Member

    @Skunnyk a fix has been accepted by maturin in PyO3/maturin#2762 that will go out with their 1.9.7 release but the Pex 2.59.3 release is also out and has a workaround which does not rely on the maturin fix.

    Using Pex 2.59.3 your original repro case is now fixed:

    # /tmp 
    :; rm -rf ~/.cache/pex/
    
    # /tmp 
    :; python -m venv myvenv
    
    # /tmp took 2s 
    :; source myvenv/bin/activate
    
    # /tmp via  v3.14.0 (myvenv) 
    :; pip install tokenizers==0.21.0 pex
    Collecting tokenizers==0.21.0
      Using cached tokenizers-0.21.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB)
    Collecting pex
      Downloading pex-2.59.3-py2.py3-none-any.whl.metadata (7.4 kB)
    Collecting huggingface-hub<1.0,>=0.16.4 (from tokenizers==0.21.0)
      Using cached huggingface_hub-0.35.3-py3-none-any.whl.metadata (14 kB)
    Collecting filelock (from huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
      Using cached filelock-3.19.1-py3-none-any.whl.metadata (2.1 kB)
    Collecting fsspec>=2023.5.0 (from huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
      Using cached fsspec-2025.9.0-py3-none-any.whl.metadata (10 kB)
    Collecting packaging>=20.9 (from huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
      Using cached packaging-25.0-py3-none-any.whl.metadata (3.3 kB)
    Collecting pyyaml>=5.1 (from huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
      Downloading pyyaml-6.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (2.4 kB)
    Collecting requests (from huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
      Using cached requests-2.32.5-py3-none-any.whl.metadata (4.9 kB)
    Collecting tqdm>=4.42.1 (from huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
      Using cached tqdm-4.67.1-py3-none-any.whl.metadata (57 kB)
    Collecting typing-extensions>=3.7.4.3 (from huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
      Using cached typing_extensions-4.15.0-py3-none-any.whl.metadata (3.3 kB)
    Collecting hf-xet<2.0.0,>=1.1.3 (from huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
      Using cached hf_xet-1.1.10-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.7 kB)
    Collecting charset_normalizer<4,>=2 (from requests->huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
      Using cached charset_normalizer-3.4.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (36 kB)
    Collecting idna<4,>=2.5 (from requests->huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
      Using cached idna-3.10-py3-none-any.whl.metadata (10 kB)
    Collecting urllib3<3,>=1.21.1 (from requests->huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
      Using cached urllib3-2.5.0-py3-none-any.whl.metadata (6.5 kB)
    Collecting certifi>=2017.4.17 (from requests->huggingface-hub<1.0,>=0.16.4->tokenizers==0.21.0)
      Using cached certifi-2025.10.5-py3-none-any.whl.metadata (2.5 kB)
    Using cached tokenizers-0.21.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
    Using cached huggingface_hub-0.35.3-py3-none-any.whl (564 kB)
    Using cached hf_xet-1.1.10-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.2 MB)
    Downloading pex-2.59.3-py2.py3-none-any.whl (3.9 MB)
       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.9/3.9 MB 9.2 MB/s  0:00:00
    Using cached fsspec-2025.9.0-py3-none-any.whl (199 kB)
    Using cached packaging-25.0-py3-none-any.whl (66 kB)
    Downloading pyyaml-6.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (794 kB)
       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 794.2/794.2 kB 9.4 MB/s  0:00:00
    Using cached tqdm-4.67.1-py3-none-any.whl (78 kB)
    Using cached typing_extensions-4.15.0-py3-none-any.whl (44 kB)
    Using cached filelock-3.19.1-py3-none-any.whl (15 kB)
    Using cached requests-2.32.5-py3-none-any.whl (64 kB)
    Using cached charset_normalizer-3.4.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (150 kB)
    Using cached idna-3.10-py3-none-any.whl (70 kB)
    Using cached urllib3-2.5.0-py3-none-any.whl (129 kB)
    Using cached certifi-2025.10.5-py3-none-any.whl (163 kB)
    Installing collected packages: urllib3, typing-extensions, tqdm, pyyaml, pex, packaging, idna, hf-xet, fsspec, filelock, charset_normalizer, certifi, requests, huggingface-hub, tokenizers
    Successfully installed certifi-2025.10.5 charset_normalizer-3.4.3 filelock-3.19.1 fsspec-2025.9.0 hf-xet-1.1.10 huggingface-hub-0.35.3 idna-3.10 packaging-25.0 pex-2.59.3 pyyaml-6.0.3 requests-2.32.5 tokenizers-0.21.0 tqdm-4.67.1 typing-extensions-4.15.0 urllib3-2.5.0
    
    # /tmp via  v3.14.0 (myvenv) took 5s 
    :; pex -V
    2.59.3
    
    # /tmp via  v3.14.0 (myvenv) 
    :; pex --venv-repository myvenv/ --layout packed  -o mypex
    
    # /tmp via  v3.14.0 (myvenv) took 2s 
    :; pex --venv-repository myvenv/ --layout packed  -o mypex
    
    # /tmp via  v3.14.0 (myvenv) 
    :; count=0; while (( $((count+=1)) < 50 )); do echo "Attempt $count:" && pex --venv-repository myvenv/ --layout packed  -o mypex && python mypex -c 'import tokenizers; print(tokenizers.__file__)'; done
    Attempt 1:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 2:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 3:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 4:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 5:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 6:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 7:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 8:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 9:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 10:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 11:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 12:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 13:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 14:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 15:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 16:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 17:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 18:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 19:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 20:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 21:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 22:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 23:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 24:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 25:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 26:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 27:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 28:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 29:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 30:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 31:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 32:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 33:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 34:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 35:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 36:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 37:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 38:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 39:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 40:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 41:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 42:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 43:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 44:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 45:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 46:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 47:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 48:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    Attempt 49:
    /home/jsirois/.cache/pex/installed_wheels/0/96bb4c870025caa1ce59de22cacb3b8f04c056246eecbf33563433929a185839/tokenizers-0.21.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl/tokenizers/__init__.py
    
    # /tmp via  v3.14.0 (myvenv) took 1m15s
  11. added a commit that references this issue on Oct 7, 2025
    83cef00
  12. Skunnyk commented on Oct 8, 2025

    @Skunnyk
    Author

    Hey!

    Thank you, seems fine on my side!

    I now have another issue around safe_symlink(rel_src, dst) from pex/common.py in installed_wheels cache folder with a FileExistsError:, but I don't yet understand what is happening nor have a simple reproducer (that happens when building multiple pex from different venv, with the same pex cache folder). I'll open another issue when I have more information.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions