Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stdlib: add overloads for itertools.product() up to an arity of 15 #13492

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

muxator
Copy link

@muxator muxator commented Feb 11, 2025

Before this change, the maximum usable arity for itertools.product() was 10 (increased in 2e83e65, #12023, which raised it from 6 to 10).

Fixes #13490, albeit partially: just based on this change, a user has no way of knowing if he is hitting this limit or not.

This discoverability problem will probably be best addressed with changes in documentation or in the type checkers.

Before this change, the maximum usable arity for itertools.product() was 10
(increased in 2e83e65, python#12023, which raised it from 6 to 10).

Fixes python#13490, albeit partially: just based on this change, a user has no way of
knowing if he is hitting this limit or not.

This discoverability problem will probably be best addressed with changes in
documentation or in the type checkers.
@muxator muxator force-pushed the itertools-product-increase-arity branch from 163748f to 4e69fa4 Compare February 11, 2025 18:54

This comment has been minimized.

@AlexWaygood AlexWaygood changed the title stdlib: add overloads fot itertools.product() up to an arity of 15 stdlib: add overloads for itertools.product() up to an arity of 15 Feb 11, 2025
@muxator muxator force-pushed the itertools-product-increase-arity branch from 4e69fa4 to 7684a23 Compare February 11, 2025 19:13
@muxator
Copy link
Author

muxator commented Feb 11, 2025

Thanks for catching the typo in the PR, @AlexWaygood. The same error was in the commit message. I have force-pushed a new version with the correct commit message.

@AlexWaygood
Copy link
Member

Thanks for catching the typo in the PR, @AlexWaygood. The same error was in the commit message. I have force-pushed a new version with the correct commit message.

Thanks! Though we squash-merge all PRs, so it would have taken the PR title rather than the title of the commit anyway :-)

Copy link
Contributor

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

@JelleZijlstra
Copy link
Member

I'd rather not do this, actually—we can keep adding more overloads for this sort of thing without an obvious limit. The stubs become harder to understand and probably slower to process, error messages become harder to understand, etc. 10 seems like a good round number to stick with.

@muxator
Copy link
Author

muxator commented Feb 12, 2025

Hi @JelleZijlstra, I understand your point of view: increasing the number of hardcoded overloads seems akin to putting sleep() calls in async code to "fix" synchronization problems. Not pretty.

Please, also consider that this request is coming form a user that is trying to use type hinting on actual code.

Calling itertools.product() with more than 10 parameters is semantically well-defined in Python and in its type system. It's not beautiful, but I suspect it is a common necessity in my niche (scientific simulation that have to cover a highly-dimensional parameter space).

Probably not many users would bother debugging and reporting these sort of problems, so please consider that the amount of communication that reaches the maintainers may be less than the actual users' need.

On a technical level, is there a way I can help to address your concerns?

  • "The stubs become harder to understand": itertools.pyi is not completely devoid of comments. Explaining the reason for the presence of many overloads would help? I can add the comments, even if you decide to stick to 10 overloads;
  • "probably slower to process": I can't reliably comment on this. I suspect it heavily depends on how the type checkers are implemented; I do not know if mypy or pyright internally perform a linear search or a O(1) lookup;
  • "error messages become harder to understand": in this case the error messages were erroneous because of the lack of enough overloads (see for example maximum arity of itertools.product() is hardcoded to 10. document, increase, or print a warning? #13490 (comment)). Maybe in other cases the messages can become worse: I trust your judgement here.

If you want some modifications to the PR (comments? tests?) I'd be happy to make them. I also think that, later, there could be a followup for trying to generate proper unambiguous error messages for pathological cases.

Let me know your opinion, thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

maximum arity of itertools.product() is hardcoded to 10. document, increase, or print a warning?
3 participants