Skip to content

uv add rejects pytorch from its index when on macΒ #5182

Closed

Description

@baggiponte

I am trying to use uv to add torch to my project's dependencies, but it fails. I am on macOS14.4.1 (ARM).

Here is a minimal repro.

[project]
name = "test"
version = "0.1.0"
requires-python = ">=3.10"

Then run:

uv python pin 3.10
uv add --extra-index-url="https://download.pytorch.org/whl/cpu" -- torch

Error:

warning: `uv add` is experimental and may change without warning.
Using Python 3.10.14 interpreter at: /Users/luca/Library/Application Support/uv/python/cpython-3.10.14-macos-aarch64-none/install/bin/python3
Creating virtualenv at: .venv
Resolved 13 packages in 1.69s
error: distribution torch==2.3.1+cpu @ registry+https://download.pytorch.org/whl/cpu can't be installed because it doesn't have a source distribution or wheel for the current platform

If I use uv pip install then everything works smoothly:

uv pip install --extra-index-url="https://download.pytorch.org/whl/cpu" -- torch
Resolved 9 packages in 1.73s
Installed 9 packages in 369ms
 + filelock==3.13.1
 + fsspec==2024.2.0
 + jinja2==3.1.3
 + markupsafe==2.1.5
 + mpmath==1.3.0
 + networkx==3.2.1
 + sympy==1.12
 + torch==2.3.1
 + typing-extensions==4.9.0

Activity

  1. charliermarsh commented on Jul 18, 2024

    @charliermarsh
    Member

    This is already resolved on main but not released.

  2. charliermarsh commented on Jul 18, 2024

    @charliermarsh
    Member

    Sorry, misread the last part of the error.

  3. charliermarsh commented on Jul 18, 2024

    @charliermarsh
    Member

    Can you include the uv.lock that is generated?

  4. added
    area:lockRelated to universal resolution and locking
    on Jul 18, 2024
  5. charliermarsh commented on Jul 19, 2024

    @charliermarsh
    Member

    This one is pretty tricky... Basically, 2.3.1+cpu includes Windows and Linux wheels, but not macOS wheels. Separately, 2.3.1 (without the local version) has macOS wheels. We choose 2.3.1+cpu because we don't evaluate platform compatibility during a universal install. But then we don't end up including macOS wheels.

    One path forward: if the user selects 2.3.1, always choose the non-local version. That at least gives users some more control.

    I don't know what other options we have. Try to sniff platform compatibility on the wheel tags, at least for macOS / Windows / Linux?

  6. charliermarsh commented on Jul 19, 2024

    @charliermarsh
    Member
  7. charliermarsh commented on Jul 19, 2024

    @charliermarsh
    Member

    I'd suggest using a direct URL for now while we figure out how to support this @baggiponte.

  8. charliermarsh commented on Jul 19, 2024

    @charliermarsh
    Member

    I've confirmed that Poetry has the same problem. It also chooses:

    [[package]]
    name = "torch"
    version = "2.3.1+cpu"
    description = "Tensors and Dynamic neural networks in Python with strong GPU acceleration"
    optional = false
    python-versions = ">=3.8.0"
    files = [
        {file = "torch-2.3.1+cpu-cp310-cp310-linux_x86_64.whl", hash = "sha256:d679e21d871982b9234444331a26350902cfd2d5ca44ce6f49896af8b3a3087d"},
        {file = "torch-2.3.1+cpu-cp310-cp310-win_amd64.whl", hash = "sha256:500bf790afc2fd374a15d06213242e517afccc50a46ea5955d321a9a68003335"},
        {file = "torch-2.3.1+cpu-cp311-cp311-linux_x86_64.whl", hash = "sha256:a272defe305dbd944aa28a91cc3db0f0149495b3ebec2e39723a7224fa05dc57"},
        {file = "torch-2.3.1+cpu-cp311-cp311-win_amd64.whl", hash = "sha256:d2965eb54d3c8818e2280a54bd53e8246a6bb34e4b10bd19c59f35b611dd9f05"},
        {file = "torch-2.3.1+cpu-cp312-cp312-linux_x86_64.whl", hash = "sha256:2141a6cb7021adf2f92a0fd372cfeac524ba460bd39ce3a641d30a561e41f69a"},
        {file = "torch-2.3.1+cpu-cp312-cp312-win_amd64.whl", hash = "sha256:6acdca2530462611095c44fd95af75ecd5b9646eac813452fe0adf31a9bc310a"},
        {file = "torch-2.3.1+cpu-cp38-cp38-linux_x86_64.whl", hash = "sha256:cab92d5101e6db686c5525e04d87cedbcf3a556073d71d07fbe7d1ce09630ffb"},
        {file = "torch-2.3.1+cpu-cp38-cp38-win_amd64.whl", hash = "sha256:dbc784569a367fd425158cf4ae82057dd3011185ba5fc68440432ba0562cb5b2"},
        {file = "torch-2.3.1+cpu-cp39-cp39-linux_x86_64.whl", hash = "sha256:a3cb8e61ba311cee1bb7463cbdcf3ebdfd071e2091e74c5785e3687eb02819f9"},
        {file = "torch-2.3.1+cpu-cp39-cp39-win_amd64.whl", hash = "sha256:df68668056e62c0332e03f43d9da5d4278b39df1ba58d30ec20d34242070955d"},
    ]

    And then poetry install fails after poetry lock succeeds.

  9. baggiponte commented on Jul 19, 2024

    @baggiponte
    ContributorAuthor

    Can you include the uv.lock that is generated?

    Ops, sorry for the delay. Here it is (sorry for the .txt format, github does not support it).

    uv.lock.txt

    I'd suggest using a direct URL for now while we figure out how to support this @baggiponte.

    i.e. from pypi, right?

    Thank you very much for the prompt support!

  10. dimbleby commented on Jul 19, 2024

    @dimbleby

    FWIW over in poetry I have mostly come to think that this is a torch problem and not a poetry problem at all. (And therefore also not a uv problem).

    Specifically I reckon that pytorch/pytorch#110004 is the right request to make, and the right place to make it. "Closing as implemented" seems not correct, perhaps someone would like to try again.

    More generally... pytorch does lots of unusual things with its multiple indexes and playing games with local version identifiers and suchlike. If you want a taste of the sort of fun uv is likely to have trying to accommodate all of this - and if you have plenty of time to spare - then see python-poetry/poetry#6409.

    If uv - either deliberately or accidentally - were to make choices that somehow made sense of all of that, then there are plenty of folk in that thread who would likely be pleased to see it!

  11. removed their assignment
    on Jul 30, 2024
  12. baggiponte commented on Aug 3, 2024

    @baggiponte
    ContributorAuthor

    Uhm as an FYI: today I tried uv add torch on M3 mac and it correctly installed the version w/o CUDA (which was brought me to use uv add --extra-index-url="https://download.pytorch.org/whl/cpu" -- torch in the first place).

  13. dimbleby commented on Sep 19, 2024

    @dimbleby
  14. charliermarsh commented on Dec 20, 2024

    @charliermarsh
    Member

    Fixed by #10046.

  15. meliz19 commented on Jan 22, 2025

    @meliz19

    I'm seeing a similar error when trying to install the Pantsbuild libraries via uv add or uv pip install. I'm on a MacOS 14.7.1 ARM machine. Full error message below.

    margaretblack test-project % uv add https://github.com/pantsbuild/pants/releases/download/release_2.23.1/pantsbuild.pants-2.23.1-cp39-cp39-macosx_11_0_arm64.whl
    Resolved 25 packages in 2.49s
    error: distribution `pantsbuild-pants==2.23.1 @ direct+https://github.com/pantsbuild/pants/releases/download/release_2.23.1/pantsbuild.pants-2.23.1-cp39-cp39-macosx_11_0_arm64.whl` can't be installed because the binary distribution is incompatible with the current platform
    
    margaretblack test-project % uv add https://github.com/pantsbuild/pants/archive/refs/tags/release_2.23.1.tar.gz
    error: Failed to parse metadata from built wheel
      Caused by: Metadata field Name not found
    
    margaretblack test-project % uv add https://github.com/pantsbuild/pants/archive/refs/tags/release_2.23.1.zip
    error: Failed to parse metadata from built wheel
      Caused by: Metadata field Name not found
    
    margaretblack test-project % uv pip install git+https://github.com/pantsbuild/pants.git@release_2.23.0
     Updated https://github.com/pantsbuild/pants.git (6fad8c66e)
    error: Failed to parse metadata from built wheel
      Caused by: Metadata field Name not found
    
  16. charliermarsh commented on Jan 22, 2025

    @charliermarsh
    Member

    Are you sure that you're using Python 3.9? Does uv add --python 3.9 https://github.com/pantsbuild/pants/releases/download/release_2.23.1/pantsbuild.pants-2.23.1-cp39-cp39-macosx_11_0_arm64.whl fail in the same way?

    As for those other failures, I'm not quite sure what to do with them -- it seems like that's not a properly structured Python package...? Running python -m build in those source directories creates invalid wheels.

  17. meliz19 commented on Jan 22, 2025

    @meliz19

    Are you sure that you're using Python 3.9? Does uv add --python 3.9 https://github.com/pantsbuild/pants/releases/download/release_2.23.1/pantsbuild.pants-2.23.1-cp39-cp39-macosx_11_0_arm64.whl fail in the same way?

    As for those other failures, I'm not quite sure what to do with them -- it seems like that's not a properly structured Python package...? Running python -m build in those source directories creates invalid wheels.

    Running uv add --python 3.9 https://github.com/pantsbuild/pants/releases/download/release_2.23.1/pantsbuild.pants-2.23.1-cp39-cp39-macosx_11_0_arm64.whl resolved this issue! Thank you so much for the quick response @charliermarsh!

  18. charliermarsh commented on Jan 22, 2025

    @charliermarsh
    Member

    No prob!

  19. charliermarsh commented on Mar 21, 2025

    @charliermarsh
    Member

    You need to declare macOS x86 as a required environment: https://docs.astral.sh/uv/concepts/resolution/#required-environments (it’s actually the example we use in the docs, so you can copy the configuration at the bottom there)

  20. Mathemmagician commented on Apr 10, 2025

    @Mathemmagician

    Just wanted to spell out the Intel x86_64 macOS solution that worked for me

    1. In pyproject.toml, add:
    [tool.uv]
    required-environments = [
        "sys_platform == 'darwin' and platform_machine == 'x86_64'"
    ]
    
    1. Downgrade NumPy to below 2.0.0:
    "numpy<2.0.0",
    
    1. Run uv sync
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:lockRelated to universal resolution and lockingbugSomething isn't working

    Type

    No type

    Projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions