Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

can not build using pip python3.10.6 #87

Open
zba opened this issue Jul 18, 2023 · 18 comments
Open

can not build using pip python3.10.6 #87

zba opened this issue Jul 18, 2023 · 18 comments

Comments

@zba
Copy link

zba commented Jul 18, 2023

Hi, when installing using pip3 getting the error
My env:
Docker
python3.10.6 is default python version for ubuntu22.04
Docker image: nvidia/cuda:11.8.0-devel-ubuntu22.04

tried to upgrade python to 3.10.8 using pyenv, same result as with default

The error

Failed to build pyworld

STDERR:
  error: subprocess-exited-with-error

  × Building wheel for pyworld (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [107 lines of output]
      /tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/dist.py:509: InformationOnly: Normalizing 'v0.3.3' to '0.3.3'
        self.metadata.version = self._normalize_version(
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-310
      creating build/lib.linux-x86_64-cpython-310/pyworld
      copying pyworld/__init__.py -> build/lib.linux-x86_64-cpython-310/pyworld
      running build_ext
      /tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /tmp/pip-install-ip2kf5fu/pyworld_c2d8c13f6eea484bbb08c586e14e2a7f/pyworld/pyworld.pyx
        tree = Parsing.p_module(s, pxd, full_module_name)
      Compiling pyworld/pyworld.pyx because it changed.
      [1/1] Cythonizing pyworld/pyworld.pyx
      Traceback (most recent call last):
        File "/install/venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/install/venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/install/venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 416, in build_wheel
          return self._build_with_temp_dir(['bdist_wheel'], '.whl',
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 401, in _build_with_temp_dir
          self.run_setup()
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 487, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in run_setup
          exec(code, locals())
        File "<string>", line 27, in <module>
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 107, in setup
          return distutils.core.setup(**attrs)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 343, in run
          self.run_command("build")
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 131, in run
          self.run_command(cmd_name)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 84, in run
          _build_ext.run(self)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
          self.build_extensions()
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
          self._build_extensions_serial()
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
          _build_ext.build_extension(self, ext)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/Cython/Distutils/build_ext.py", line 122, in build_extension
          new_ext = cythonize(
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1134, in cythonize
          cythonize_one(*args)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1280, in cythonize_one
          result = compile_single(pyx_file, options, full_module_name=full_module_name)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/Cython/Compiler/Main.py", line 615, in compile_single
          return run_pipeline(source, options, full_module_name)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/Cython/Compiler/Main.py", line 539, in run_pipeline
          err, enddata = Pipeline.run_pipeline(pipeline, source)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/Cython/Compiler/Pipeline.py", line 398, in run_pipeline
          data = run(phase, data)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/Cython/Compiler/Pipeline.py", line 375, in run
          return phase(data)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/Cython/Compiler/Pipeline.py", line 52, in generate_pyx_code_stage
          module_node.process_implementation(options, result)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/Cython/Compiler/ModuleNode.py", line 222, in process_implementation
          self.generate_c_code(env, options, result)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/Cython/Compiler/ModuleNode.py", line 490, in generate_c_code
          self.generate_module_preamble(env, options, modules, result.embedded_metadata, h_code)
        File "/tmp/pip-build-env-2uhekrvj/overlay/lib/python3.10/site-packages/Cython/Compiler/ModuleNode.py", line 773, in generate_module_preamble
          code.putln(json.dumps(metadata, indent=4, sort_keys=True))
        File "/usr/lib/python3.10/json/__init__.py", line 238, in dumps
          **kw).encode(obj)
        File "/usr/lib/python3.10/json/encoder.py", line 201, in encode
          chunks = list(chunks)
        File "/usr/lib/python3.10/json/encoder.py", line 431, in _iterencode
          yield from _iterencode_dict(o, _current_indent_level)
        File "/usr/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict
          yield from chunks
        File "/usr/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict
          yield from chunks
        File "/usr/lib/python3.10/json/encoder.py", line 325, in _iterencode_list
          yield from chunks
        File "/usr/lib/python3.10/json/encoder.py", line 438, in _iterencode
          o = _default(o)
        File "/usr/lib/python3.10/json/encoder.py", line 179, in default
          raise TypeError(f'Object of type {o.__class__.__name__} '
      TypeError: Object of type PosixPath is not JSON serializable
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pyworld
ERROR: Could not build wheels for pyworld, which is required to install pyproject.toml-based projects
@zba zba mentioned this issue Jul 18, 2023
Closed
@ota
Copy link

ota commented Jul 18, 2023

Hi, I also encountered the same issue in a Python 3.10.12 conda environment and resolved it by following the installation steps below:
pip install numpy
pip install pyworld --no-build-isolation

@jinzhen-lin
Copy link

It seems that the reason is cython released v3.0.0 yesterday.

@rmcpantoja
Copy link

Hi, same problem in colab:
error: subprocess-exited-with-error

× Building wheel for pyworld (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
Building wheel for pyworld (pyproject.toml) ... error
ERROR: Failed building wheel for pyworld

@JeremyCCHsu
Copy link
Owner

Thank you @ota for the solution.
Does anyone know how to resolve this issue? PRs are welcome!

@264768502
Copy link

limit cython version in pyproject.toml for workaround

@maratsubkhankulov maratsubkhankulov mentioned this issue Jul 18, 2023
Closed
@x6rulin
Copy link

x6rulin commented Jul 19, 2023

It seems that this issue occurs only when using pip install pyworld, while there are no exceptions when running pip install . in pyworld root directory.

@x6rulin
Copy link

x6rulin commented Jul 19, 2023 •
edited
Loading

It seems that the reason is cython released v3.0.0 yesterday.

It is unlikely caused by Cython v3.0.0. There are no exceptions encountered when directly running pip install ., and pip install pyworld with cython<=0.29.36 still fails.

@jinzhen-lin
Copy link

It seems that the reason is cython released v3.0.0 yesterday.

It is unlikely caused by Cython v3.0.0. There are no exceptions encountered when directly running pip install ., and pip install pyworld with cython<=0.29.36 still fails.

This project use pyproject.toml, so when you run pip install pyworld, it would build in a isolated python environment. In this environment, the latest cython is installed (no matter which cython version you install in your python environment).

When you add --no-build-isolation to pip or just run pip install ., the project would build in current python environment.

@x6rulin
Copy link

x6rulin commented Jul 19, 2023 •
edited
Loading

It seems that the reason is cython released v3.0.0 yesterday.

It is unlikely caused by Cython v3.0.0. There are no exceptions encountered when directly running pip install ., and pip install pyworld with cython<=0.29.36 still fails.

This project use pyproject.toml, so when you run pip install pyworld, it would build in a isolated python environment. In this environment, the latest cython is installed (no matter which cython version you install in your python environment).

When you add --no-build-isolation to pip or just run pip install ., the project would build in current python environment.

Thanks for the explanation about --no-build-isolation. I need to supplement that pip install . is done with Cython v3.0.0.

With only numpy pre-installed, pip install pyworld --no-build-isolation will execute successfully and install Cython v3.0.0 simultaneously.

@x6rulin
Copy link

x6rulin commented Jul 19, 2023

I debugged it. The inclusion of a Path object (PosixPath('lib/World/src')) in the include_dirs list when calling cythonize causes the json.dump exception. But I can't figure out how it comes there.

@rmcpantoja
Copy link

Just a note: colab is still using cyton 0.29.36.

@JeremyCCHsu JeremyCCHsu mentioned this issue Jul 19, 2023
Merged
@JeremyCCHsu
Copy link
Owner

It appears to be a pyproject.toml issue. numpy should be installed before cython (thx to @ota 's hint). Automatic update will be done after the tests are done. Please try pip install, say, 20 minute later. (I'm sure in #89 will at least make local installation possible.)

@JeremyCCHsu
Copy link
Owner

plz build pyworld from source following this for now... I can't get Travis CI to work properly and thus PYPI isn't updated. I'll figure it out later.

@wang01250
Copy link

you can build form soure and modify the requirments.txt with "cpython==0.29.36"

@JeremyCCHsu JeremyCCHsu mentioned this issue Jul 20, 2023
Closed
@kan-bayashi kan-bayashi mentioned this issue Jul 20, 2023
Merged
@gakugaku gakugaku mentioned this issue Jul 20, 2023
Closed
@JeremyCCHsu
Copy link
Owner

JeremyCCHsu commented Jul 20, 2023 •
edited
Loading

I've manually uploaded v0.3.4. I tested pip install pyworld on a clean-slate env, and it worked. Please let me know if pip install is still failing on your Linux systems.

For Windows users, I'm really sorry, but PyPI's new 2FA policy makes it impossible to allow my CI (Travis/Appveyor) to upload automatically upload the .whl files (I'm already using PyPI tokens). If you know how to get Travis/Appveyor to upload the artifacts automatically under the 2FA constraint, plz lmk. I'll really appreciate it. I've been seeing the following error message for thousands of times...
(.whl files are uploaded now.)

@sweetbbak
Copy link

I just tried an hour ago and it failed and I ended finding this issue.

@mitchsayre
Copy link

mitchsayre commented Aug 5, 2023 •
edited
Loading

(Note that I am using python3.9 and ubuntu20.04 image but I believe it is the same issue)
I got the package to install correctly in Docker by adding this to my Dockerfile apt-get install -y build-essential python-dev python3-dev python3.9-distutils python3.9-dev python3.9 seems like one of pyworld's dependencies now requires a python-dev header I am not sure which ones of these apt packages actually made a difference. Here is my full Dockerfile:

FROM nvidia/cuda:11.6.2-cudnn8-runtime-ubuntu20.04

# Install dependenceis to add PPAs
RUN apt-get update && \
    apt-get install -y software-properties-common && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Add the deadsnakes PPA to get Python 3.9
RUN add-apt-repository ppa:deadsnakes/ppa

# Install Python 3.9 and pip
RUN apt-get update && \
    apt-get install -y build-essential python-dev python3-dev python3.9-distutils python3.9-dev python3.9 curl && \
    apt-get clean && \
    update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 1 && \
    curl https://bootstrap.pypa.io/get-pip.py | python3.9

# Set Python 3.9 as the default
RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.9 1

# Set the working directory
WORKDIR /app

# Copy application files to the container
COPY . .

# Install python dependencies
RUN pip install -r requirements.txt

CMD python main.py

Also here are the package versions I targeted in my requirements.txt:

numpy==1.23.5
pyworld==0.3.2

@zba
Copy link
Author

zba commented Sep 15, 2023

Try to install pyword using RUN separately from requirements.txt, as described above

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
10 participants
@zba @ota @264768502 @jinzhen-lin @JeremyCCHsu @x6rulin @mitchsayre @rmcpantoja @wang01250 @sweetbbak