Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to use external dependency with uniffi #2459

Open
1 of 2 tasks
dignifiedquire opened this issue Jan 29, 2025 · 1 comment
Open
1 of 2 tasks

Unable to use external dependency with uniffi #2459

dignifiedquire opened this issue Jan 29, 2025 · 1 comment
Labels
bindings/uniffi uniffi bindings bug Something isn't working

Comments

@dignifiedquire
Copy link

dignifiedquire commented Jan 29, 2025

Bug Description

I am trying to use an external package in python bindings, and it almost works, but not quite unfortunately.

  • package a: iroh-ffi
  • package b: iroh-ffi-gossip
# iroh-ffi/Cargo.toml
[package]
name = "iroh-ffi"

# ..

[lib]
name = "iroh_ffi"
crate-type = ["staticlib", "cdylib", "rlib"]

[package.metadata.maturin]
name = "iroh"
# iroh-ffi/uniffi.toml
[bindings.python]
cdylib_name = "iroh_ffi"
# iroh-gossip/Cargo.toml

[package]
name = "iroh-ffi-gossip"

# ..

[package.metadata.maturin]
name = "iroh_gossip"

[lib]
name = "iroh_ffi_gossip"
crate-type = ["staticlib", "cdylib"]
# iroh-gossip/uniffi.toml
[bindings.python]
cdylib_name = "iroh_ffi_gossip"

[bindings.python.external_packages]
iroh_ffi = "iroh"

When I now generate the bindings using maturin, I end up with the following __init__.py

from .iroh_ffi import *  # NOQA
from .iroh_ffi_gossip import *  # NOQA

as well as the following files

ls .venv/lib/python3.13/site-packages/iroh_gossip/
__init__.py              iroh_ffi.py              libiroh_ffi_gossip.dylib
__pycache__              iroh_ffi_gossip.py

This will fail, because there is no matching dylib for iroh_ffi, so running this will explode. Commenting out the from .iroh_ffi import * fixes the issue, as all imports in iroh_ffi_gossip.py are correctly pointing to iroh.

Now what I would like to have is to simply depend on the iroh python package, but due to the unecessary inclusions of its ffi code the generation is currently broken

Your maturin version (maturin --version)

1.8.1

Your Python version (python -V)

3.13.1

Your pip version (pip -V)

24.3.1

What bindings you're using

uniffi

Does cargo build work?

  • Yes, it works

If on windows, have you checked that you aren't accidentally using unix path (those with the forward slash /)?

  • Yes
@dignifiedquire dignifiedquire added the bug Something isn't working label Jan 29, 2025
@messense messense added the bindings/uniffi uniffi bindings label Feb 5, 2025
@messense
Copy link
Member

messense commented Feb 5, 2025

We don't have lots of usage of uniffi binding so support can be rough, pull requests are welcome if you'd like to improve it, thanks! The related code is in here:

maturin/src/module_writer.rs

Lines 1181 to 1267 in 2861bf1

/// Creates the uniffi module with the shared library
#[allow(clippy::too_many_arguments)]
pub fn write_uniffi_module(
writer: &mut impl ModuleWriter,
project_layout: &ProjectLayout,
crate_dir: &Path,
target_dir: &Path,
module_name: &str,
artifact: &Path,
target_os: Os,
editable: bool,
pyproject_toml: Option<&PyProjectToml>,
) -> Result<()> {
let UniFfiBindings {
names: binding_names,
cdylib,
path: binding_dir,
} = generate_uniffi_bindings(crate_dir, target_dir, module_name, target_os, artifact)?;
let py_init = binding_names
.iter()
.map(|name| format!("from .{name} import * # NOQA\n"))
.collect::<Vec<String>>()
.join("");
if !editable {
write_python_part(writer, project_layout, pyproject_toml)
.context("Failed to add the python module to the package")?;
}
let module;
if let Some(python_module) = &project_layout.python_module {
if editable {
let base_path = python_module.join(&project_layout.extension_name);
fs::create_dir_all(&base_path)?;
let target = base_path.join(&cdylib);
fs::copy(artifact, &target).context(format!(
"Failed to copy {} to {}",
artifact.display(),
target.display()
))?;
File::create(base_path.join("__init__.py"))?.write_all(py_init.as_bytes())?;
for binding_name in binding_names.iter() {
let target: PathBuf = base_path.join(binding_name).with_extension("py");
fs::copy(binding_dir.join(binding_name).with_extension("py"), &target)
.with_context(|| {
format!("Failed to copy {:?} to {:?}", binding_dir.display(), target)
})?;
}
}
let relative = project_layout
.rust_module
.strip_prefix(python_module.parent().unwrap())
.unwrap();
module = relative.join(&project_layout.extension_name);
if !editable {
writer.add_directory(&module)?;
}
} else {
module = PathBuf::from(module_name);
writer.add_directory(&module)?;
let type_stub = project_layout
.rust_module
.join(format!("{module_name}.pyi"));
if type_stub.exists() {
eprintln!("📖 Found type stub file at {module_name}.pyi");
writer.add_file(module.join("__init__.pyi"), type_stub)?;
writer.add_bytes(module.join("py.typed"), None, b"")?;
}
};
if !editable || project_layout.python_module.is_none() {
writer.add_bytes(module.join("__init__.py"), None, py_init.as_bytes())?;
for binding in binding_names.iter() {
writer.add_file(
module.join(binding).with_extension("py"),
binding_dir.join(binding).with_extension("py"),
)?;
}
writer.add_file_with_permissions(module.join(cdylib), artifact, 0o755)?;
}
Ok(())
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bindings/uniffi uniffi bindings bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants