Skip to content

Importing mixed Rust/Python project fails when structured as recommended #490

Closed
frank1010111/pywaterflood
#62

Description

@ArniDagur

The README of this project says the following:

Mixed rust/python projects

To create a mixed rust/python project, create a folder with your module name (i.e. lib.name in Cargo.toml) next to your Cargo.toml and add your python sources there:

my-project
├── Cargo.toml
├── my_project
│   ├── __init__.py
│   └── bar.py
├── Readme.md
└── src
    └── lib.rs

maturin will add the native extension as a module in your python folder. When using develop, maturin will copy the native library and for cffi also the glue code to your python folder. You should add those files to your gitignore.

With cffi you can do from .my_project import lib and then use lib.my_native_function, with pyo3/rust-cpython you can directly from .my_project import my_native_function.

Example layout with pyo3 after maturin develop:

my-project
├── Cargo.toml
├── my_project
│   ├── __init__.py
│   ├── bar.py
│   └── my_project.cpython-36m-x86_64-linux-gnu.so
├── Readme.md
└── src
    └── lib.rs

This is the structure I'm using for python-adblock. The project is written entirely in Rust, but I'm using a mixed Rust/Python project in order to include Python type stubs. I have an __init__.py which imports the neccesary symbols from the native module.

However, some of my users have reported that the imports inside __init__.py fail as follows:

>>> import adblock
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/app/adblock/__init__.py", line 1, in <module>
    from .adblock import __version__, Engine, FilterSet, BlockerResult, UrlSpecificResources
ModuleNotFoundError: No module named 'adblock.adblock'

or

>>> import adblock
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/app/adblock/__init__.py", line 1, in <module>
    from adblock.adblock import __version__, Engine, FilterSet, BlockerResult, UrlSpecificResources
ModuleNotFoundError: No module named 'adblock.adblock'

See ArniDagur/python-adblock#17. As you can see, the result is the same whether I use from .adblock import ... or from adblock.adblock import .... I have only managed to reproduce this when the package is installed system wide (pip3 install .) and the current working directory is the root of my project (git repository).

Does anyone know what is going on? Am I doing something different from what is described in the README.md, or is the advice in the README.md wrong in some way?

Activity

  1. omnivagant commented on Apr 10, 2021

    @omnivagant
  2. messense commented on Apr 11, 2021

    @messense
  3. messense commented on Apr 11, 2021

    @messense
  4. omnivagant commented on Apr 11, 2021

    @omnivagant
  5. omnivagant commented on Apr 11, 2021

    @omnivagant
  6. messense commented on Apr 12, 2021

    @messense
  7. omnivagant commented on Apr 12, 2021

    @omnivagant
  8. messense commented on Apr 12, 2021

    @messense
  9. messense commented on Apr 12, 2021

    @messense
  10. konstin commented on Apr 12, 2021

    @konstin
  11. omnivagant commented on Apr 12, 2021

    @omnivagant
  12. omnivagant commented on Apr 12, 2021

    @omnivagant
  13. messense commented on Apr 13, 2021

    @messense
  14. messense commented on May 1, 2021

    @messense
  15. omnivagant commented on May 12, 2021

    @omnivagant
  16. 17 remaining items

  17. added a commit that references this issue on Aug 1, 2024
  18. oyvindln commented on Jan 11, 2025

    @oyvindln

    In case anyone else ends up here like me and have a project in transition where it's not practical to move the python source into a subfolder yet - adding the python packages into pyproject.toml like so seems to help make them actually be included in the wheel and thus be accessible.

    python-packages = ["package1", "package2"]
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions