Implement Metadata 2.3

[groodt]

PEP 643 introduces Metadata 2.2, where fields can either be static (default) or marked as dynamic.
PEP 685 introduces Metadata 2.3, that specifies how to normalize extra names.

This PR bumps the Metadata version to 2.3. flit has always written and published static metadata in both wheel METADATA and sdist PKG-INFO and therefore this PR does not intend to add new support for recording dynamic metadata into distributions. This should not be confused with the dynamic metadata in the pyproject.toml files, which are dynamic in flit and remain supported (PEP 621).

Upgrading to the newer Core Metadata standards has advantages for the python packaging ecosystem. It makes it possible for resolvers to read metadata directly without a PEP 517 invocation, which can bring large speedups.

Towards: #675

[groodt]

Thinking about this more, I think I’m definitely over normalizing Requires-Dist if there are any markers. I’ll find some time to change it so that it only normalizes any extras.

[groodt]

I've added some test cases. I believe that this is now ready for review.

[takluyver]

Thanks, this is looking good to me. 🙂

One minor wrinkle is that if we fall back to executing the module to get version & description, we can't actually guarantee that they won't have a different value if you build a wheel from the sdist. You could theoretically construct them so they differ based on the day of the week, or the system it's built on. But the spec also says version can't be dynamic, and hopefully no-one actually does that, so let's cross our fingers and just say it's static.

I wonder if it would make sense to normalise values on the way in to Metadata, so checking e.g. md.provides_extra == other.provides_extra uses the normalised values. But I don't think we do any comparisons like that anywhere in the code, so let's leave it as you've done it unless we see a concrete reason to change.

[takluyver]

Thanks @groodt !

[groodt]

My pleasure! Although, I think this might have been merged prematurely. I should have moved it to draft. I have doubts over how I currently implemented it. See https://discuss.python.org/t/status-or-funding-for-metadata-2-2-in-pypi/46137/36

…

On Wed, 1 May 2024 at 05:12, Thomas Kluyver ***@***.***> wrote: Thanks @groodt <https://github.com/groodt> ! — Reply to this email directly, view it on GitHub <#676 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACT254MJBATAJNQ7PCYK33Y77UIRAVCNFSM6AAAAABEPRUYKGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBWGY4TINBVHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[takluyver]

Ah, good point. Sorry, I didn't notice your message until now.

My reading of both PEP 685 and the core metadata spec is that they only explicitly say the [extra,names] you require of another project have to be normalised. But I think the intent is that optional dependencies with an extra='foo' environment marker should probably be normalised too, and it seems best to do so. Would you be interested in making a follow-up PR to do that?

Looking at PEP 685, we should also make sure we throw suitable errors if invalid or clashing extras are specified.

[groodt]

@takluyver Yes, I'm happy to pick it up as a follow-up. Are we sure that I'm not breaking anything with the currently merged code?

I guess I'm not familiar enough with the 2 styles of extras from a library author point of view. I use extras often when I'm building apps, but not when publishing libraries as wheels.

Are these both valid Requires-Dist entries? It's unclear from the PEPs I've read.

• Requires-Dist: foo[bar]
• Requires-Dist: foo; extra == 'bar'

Are they semantically different?

[merwok]

requires foo[bar] means: I want foo, with its extra bar (and the dependencies for both)

requires foo; extra == bar means: I want foo, if I am installed with my extra bar

[groodt]

Thanks @merwok That makes sense!

[takluyver]

I'd be surprised if things break with the current state - I'd guess pip & other package consumer tools will normalise extras in environment markers (extra == 'bar') anyway, because even if they read the spec differently, they need to handle packages with older metadata versions, and it's easier to normalise in all cases than check the metadata version first.

But to be on the safe side, I'd ideally like to normalise those as well before releasing flit_core claiming to produce metadata 2.3. I don't see any downside to making extras in environment markers consistent with Provides-Extra fields.