use of setup_keywords and pyproject.toml

[pelson]

I have a need for custom metadata in a number of my projects (to declare a set of dependencies on Java packages, and have those handled by a common library). The metadata unfortunately doesn't fit as an entry point, but the docs at https://setuptools.pypa.io/en/latest/userguide/extension.html#adding-new-egg-info-files are really interesting in this regard.

In fact, if I follow the guidance in that link, and produce a subsequent package with a build-time dependency on the common library, plus the additional field (foo_bar in the example), everything works great. I'm interested to find out more about the longevity of this feature - for example, will it be possible to declare it in pyproject.toml under [tool.setuptools] at some point? Today, this is not working ("configuration error: tool.setuptools must not contain {'foo_bar'} properties").

[abravalheri]

Hi @pelson, please note we can only guarantee additional egg-info files on the sdist, as egg-info folders only exist in the sdist, for the time being¹. The way this translates into the wheel is not under the control of the setuptools project (so we cannot provide any assurances for that).

Regarding the distutils.setup_keywords plugin hook: its primary purpose is to add custom keyword arguments to the setup() function inside setup.py. However, as a side effect it also adds a custom attribute to the Distribution object used by setuptools and passed to the setuptools.finalize_distribution_options plugin hook.

There is no plan for setuptools itself to add custom fields to [tool.setuptools].
But you can see an example in setuptools-rust of how both entry-points mentioned above are leveraged in tandem to implement custom configuration via pyproject.toml (using a different TOML table).

Footnotes

1. In the long term vision of the setuptools project (with no foreseeable date to be implemented) is to get rid of such folders as they seem to be reminiscent of the deprecated .egg distribution format (as their name give away). ↩

[pelson]

we can only guarantee additional egg-info files on the sdist, as egg-info folders only exist in the sdist

This is great to know about, thank you!

The example of setuptools-rust looks super interesting! I will play a bit, and will probably come back with more questions. From what I can tell, I will be able to use this hook to add package data to the distribution based on the content of a dedicated pyproject.toml table. This would be good enough for this case I think.

[pelson]

I realised that in the core metadata spec there is Requires-External, which would seemingly be a perfect fit for defining the additional metadata driving my original question (declaring a dependency on some Java libraries, and writing a tool to inspect all installed distributions to resolve the full Java set of packages).

In terms of moving that idea forwards though, I met two road blocks:

1. It isn't possible (or at least obvious) where to define this in the project metadata.
2. If I try to add this metadata as a (wheel) post-processing step, e.g. via a custom cmdclass definition, I'm not clear that I can use pyproject only to define this (the command would be implemented in a build-time dependency, and used by many libraries with Java dependencies)

Does anybody have ideas about the above? And is there something else I'm overlooking in terms of approach?

(#3152 looks relevant, but can't figure out the syntax reqd)

[pelson]

I see that an earlier MR to setuptools explicitly ruled-out defining Requires-External as something configurable #631 (comment).

This leads me to the second question:

Is it possible to specify the cmndclasses in pyprojec.toml, such that I can post-process a build sdist/wheel's metadata (taking care to update the record). I guess I could do this as a build backend, but I don't want to have to impose to much on each project that will need to do this (and they could legitimately want to use another build backend I suppose).

Thoughts on this @abravalheri - ultimately, I would love to have python -m build produce a distribution with Requires-External metadata out-of-the-box.

[abravalheri]

The fields supported by pyproject.toml are documented in https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html#setuptools-specific-configuration.

I recommend adding cmdclass to setuptools is via setup.py. Note that if you want to import any code from the project, you still need to add the code directory to sys.path (using setup.py).

[abravalheri]

Maybe it is worthy to have a look on this one: https://github.com/wimglenn/setuptools-ext