Pandas cannot import numpy in free-threaded version of 3.13.0b2

[kdgts]

Bug report

Bug description:

1. Install python 3.13.0b2 on windows 10, make sure to install free-threaded binaries
2. Go to the python installation directory and install pandas by running:
   python -m pip install pandas
   This will also install numpy
3. Create file KDTest.py someplace outside of python installation directory with the following content:

from pandas import Timestamp

if __name__ == '__main__':
    t1: Timestamp = Timestamp("2023-11-29")
    print("Done")

4. Assuming python is installed to C:\apps\python313 run
   C:\apps\python313\python.exe <path to file>/KDTest.py
   Result: all good, Done is printed
5. Now run:
   C:\apps\python313\python3.13t.exe <path to file>/KDTest.py
   Result:

Traceback (most recent call last):
  File "<path to file>\KDTest.py", line 1, in <module>
    from pandas import Timestamp
  File "C:\apps\python313\Lib\site-packages\pandas\__init__.py", line 19, in <module>
    raise ImportError(
        "Unable to import required dependencies:\n" + "\n".join(_missing_dependencies)
    )
ImportError: Unable to import required dependencies:
numpy: Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and relaunch
        your python interpreter from there.

CPython versions tested on:

3.13

Operating systems tested on:

Windows

[JamesParrott]

This issue should be closed, and reopened in the Numpy and Pandas repos.

The latest release of Numpy, version 2.0.0, came out only yesterday. It does not support Python 3.13.

The Python versions supported by this release are 3.9-3.12
https://numpy.org/devdocs/release/2.0.0-notes.html

Nonetheless, Numpy support for Python 3.13 is actively being worked on: numpy/numpy#26157

@kdgts
There are only bdists on PyPi for Python 3.12 for both Pandas and Numpy,
https://pypi.org/project/pandas/#files
https://pypi.org/project/numpy/#files

Pip will have attempted to build them both, and quite likely silently failed (unless all of the following build dependencies were installed: https://numpy.org/doc/stable/building/index.html).

[colesbury]

I think there may be an underlying issue in CPython here where the packages are shared between the free-threaded and default builds in ways that lead to confusing error messages.

@kdgts wrote:

install pandas by running: python -m pip install pandas

But this is installing pandas and numpy packages for the default build (not free-threaded)

You'd want to install them by python3.13t.exe -m pip install pandas

[ngoldbaum]

There are Linux and Mac nightlies for cython, numpy and scipy at https://anaconda.org/scientific-python-nightly-wheels/repo.

Pandas support is being worked on and as soon as that is working wheels for pandas should appear there as well.

We are tracking ecosystem-wide support for free-threaded Python 3.13 here: https://github.com/Quansight-Labs/free-threaded-compatibility

[kdgts]

Well, given that everything works just fine with python13 regular but fails with python13 free-threaded it's pretty obvious that issue is not with numpy or pandas but rather with free-threaded python

[JamesParrott]

My apologies - it was taking so much longer than normal to install pandas, I assumed pip had hung. But it completed eventually.

I have reproduced the behaviour (after installing pandas into a venv first), and it is the same error as in your post, pretty much verbatim. Moreover, on simply importing numpy:

"%LOCALAPPDATA%\Programs\Python\Python313\python3.13t.exe" -c "import numpy"
Traceback (most recent call last):
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\site-packages\numpy\_core\__init__.py", line 23, in <module>
    from . import multiarray
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\site-packages\numpy\_core\multiarray.py", line 10, in
 <module>
    from . import overrides
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\site-packages\numpy\_core\overrides.py", line 8, in <module>
    from numpy._core._multiarray_umath import (
        add_docstring,  _get_implementing_args, _ArrayFunctionDispatcher)
ModuleNotFoundError: No module named 'numpy._core._multiarray_umath'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\site-packages\numpy\__init__.py", line 114, in <module>
    from numpy.__config__ import show as show_config
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\site-packages\numpy\__config__.py", line 4, in <module>
    from numpy._core._multiarray_umath import (
    ...<3 lines>...
    )
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\site-packages\numpy\_core\__init__.py", line 49, in <module>
    raise ImportError(msg)
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.13 from "%LOCALAPPDATA%\Programs\Python\Python313\python3.13t.exe"
  * The NumPy version is: "2.0.0"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: No module named 'numpy._core._multiarray_umath'

[lesteve]

For pandas and Python 3.13 free-threading (and more generally Python 3.13) you probably want to follow pandas-dev/pandas#58734. It looks like there is some work going on but that for now there are some known issues.

If you really want to start testing locally, my main advice would be to rely as much as possible on development wheels as hinted by @ngoldbaum in #120653 (comment). I am not sure there are some on Windows right now though but for Linux and macOS there is more chance. For example for numpy, you want to look at files with cp313-cp313t in their name in https://anaconda.org/scientific-python-nightly-wheels/numpy/files, similarly for other projects.

Here are some things you need to do (there may be other caveats):

• install pip prelease, at the time of writing, you need pip>=24.1.b1 to be able to find free-threaded wheels, i.e. wheels with cp313-cp313t in their name
• make sure to install development version wheels this works on Linux (numpy, scipy and very shortly scikit-learn will have a Linux free-threaded development wheel for macOS double-check, no scikit-learn wheel for now for sure). If you see that it downloads .tar.gz it will compile from source (hence the long time) and it will probably not work unless you really know what you are doing:

  pip install --pre --upgrade --extra-index https://pypi.anaconda.org/scientific-python-nightly-wheels/simple numpy scipy -v

  make sure that the pip is the one for your Python 3.13 free-threaded as mentioned in Pandas cannot import numpy in free-threaded version of 3.13.0b2 #120653 (comment)

If you really want to compile from source you need at the time of writing a cython development wheel so something like:

pip install --pre --upgrade --extra-index https://pypi.anaconda.org/scientific-python-nightly-wheels/simple cython -v

[rgommers]

I am not sure there are some on Windows right now though

Not yet, Windows is still in the works. Python 3.13.0b2 was the first version that included free-threaded support in Python itself; there is an issue with it in pyconfig.h that needs working around before NumPy can be built. I'm working on that right now actually - it'll be a few weeks at least before the fixes to the build system fixes propagate and Pandas & co can be built as well.