Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump supported cpython version to 3.14 for testing #4811

Open
wants to merge 16 commits into
base: main
Choose a base branch
from

Conversation

clin1234
Copy link

This is to mainly silence this abominable error:

        error: the configured Python interpreter version (3.14) is newer than PyO3's maximum supported version (3.13)
        = help: please check if an updated version of PyO3 is available. Current version: 0.23.1

@davidhewitt
Copy link
Member

Thanks for the PR. I am not comfortable bumping this check at this point. If we did, unsuspecting users will likely install software built with the next PyO3 version (call it 0.24) when 3.14 stable lands... and they will likely have a terrible experience with random crashes and other frustrating hard-to-debug and dangerous issues.

At the same time, I appreciate the desire to get testing with 3.14 and that PyO3 should not be a blocker for this. See #4662 (comment)
for my latest suggestion on what to do here.

We also have an undocumented environment variable UNSAFE_PYO3_SKIP_VERSION_CHECK=1 which can be used for development / testing. We could document that option, though with the heavy caveat that it's highly likely any unsupported version of Python will crash pretty quickly.

If you are testing PyO3 with 3.14 you may need to make patches to pyo3-ffi to match any changes in the current alphas. I think I'm ok with having such patches in PyO3 to allow testing to make progress, though of course we will not guarantee any stability until at least the beta versions land.

@clin1234
Copy link
Author

clin1234 commented Jan 17, 2025

To investigate:

  • Following tests fail in 3.14 with Rust stable in the form akin to
internal error: entered unreachable code
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/9fc6b43126469e3858e2fe86cafb4f0fd5068869\library/std\src\panicking.rs:665
   1: core::panicking::panic_fmt
             at /rustc/9fc6b43126469e3858e2fe86cafb4f0fd5068869\library/core\src\panicking.rs:76
   2: core::panicking::panic
             at /rustc/9fc6b43126469e3858e2fe86cafb4f0fd5068869\library/core\src\panicking.rs:148
   3: pyo3::instance::Borrowed<pyo3::types::string::PyString>::data
             at .\src\types\string.rs:408
   4: pyo3::types::string::impl$2::data
             at .\src\types\string.rs:313
   5: pyo3::types::string::tests::test_string_data_ucs1::closure$0
             at .\src\types\string.rs:676
   6: pyo3::marker::Python::with_gil<pyo3::types::string::tests::test_string_data_ucs1::closure_env$0,tuple$<> >
             at .\src\marker.rs:412
   7: pyo3::types::string::tests::test_string_data_ucs1
             at .\src\types\string.rs:674
   8: pyo3::types::string::tests::test_string_data_ucs1::closure$0
             at .\src\types\string.rs:673
   9: core::ops::function::FnOnce::call_once<pyo3::types::string::tests::test_string_data_ucs1::closure_env$0,tuple$<> >
             at C:\Users\runneradmin\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ops\function.rs:250
  10: core::ops::function::FnOnce::call_once
             at /rustc/9fc6b43126469e3858e2fe86cafb4f0fd5068869\library/core\src\ops\function.rs:250
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
  • types::string::tests::test_string_data_ucs1
  • types::string::tests::test_string_data_ucs1_invalid
  • types::string::tests::test_string_data_ucs2
  • types::string::tests::test_string_data_ucs2_invalid
  • types::string::tests::test_string_data_ucs4
  • types::string::tests::test_string_data_ucs4_invalid
  • clippy failing on beta Rust toolchains with
   error: this looks like a formatting argument but it is not part of a formatting macro
   --> src/lib.rs:3:22
    |
  3 |       feature = "nightly",
    |  ______________________^
  4 | |     feature(auto_traits, negative_impls, try_trait_v2)
    | |_^
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#literal_string_with_formatting_args
    = note: `-D clippy::literal-string-with-formatting-args` implied by `-D warnings`
    = help: to override `-D warnings` add `#[allow(clippy::literal_string_with_formatting_args)]`

@clin1234
Copy link
Author

@davidhewitt Might need your set of eyeballs, something has changed significantly in regards to 3.14 with strings representation(?)

@davidhewitt
Copy link
Member

This is a great example of the kind of churn which premature 3.14 support will burden us with.

The layout of PyUnicodeObject will have changed internally, so I suspect that PyUnicode::data is now UB.

Personally I'm disinterested in fixing it for an alpha which might yet churn again, I suggest just disabling this API on 3.14 with a note that we would prefer not to reintroduce it.

@ZeroIntensity
Copy link

It's great to test in alphas, but PyO3 probably isn't one of the projects that should do that, at least for the non-stable APIs; we mess with the internals all the time in alphas. I'd wait until the beta freeze (or to at least get close to it) before trying to test 3.14.

I also personally don't think PyO3 support for it would help catch too many bugs:

  • Bugs tend to be in the new features, not the old ones that have already been tested. Unless you're planning on trying to also add PyO3 support for all the new C API functions (which I really don't recommend), then it's unlikely that the early birds will catch that much. (And for the minority that do catch regressions, it's probably not worth the burden on PyO3.)
  • The other issue is that there's a bit of a programming language barrier between Rust developers and C(Python) developers. There have been several Rust-related reports in the past where it's difficult to tell whether something is a user-error, a PyO3 issue, or an actual core problem. I suspect introducing PyO3 support for the 3.14 alphas will cause quite a bit of noise, and we just don't have enough Rust experts on our end to efficiently triage all the issues.
  • Tangentially related to that point, I don't know how much PyO3 exposes implementation details. If you guys expose/use some private APIs on your end, we're bound to get reports about it if we break them.

All that said, it might be worth trying to support 3.14 early for the limited API only, because we have to ensure stability for that anyway. I don't know how feasible that is for you guys, though.

When `rustc` is invoked, the macro is included with the `--check-cfg`
flag, but not with the `--cfg` flag. This caused errors about duplicate
definitions to spew out when building with stable Rust toolchains.
@clin1234 clin1234 force-pushed the main branch 2 times, most recently from 3b49269 to 4c94698 Compare January 23, 2025 20:07
@clin1234
Copy link
Author

CI passes, might need to consolidate commits

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants