Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Parallel execution needs serialization #1423

Open
HaoZeke opened this issue Aug 12, 2024 · 0 comments
Open

BUG: Parallel execution needs serialization #1423

HaoZeke opened this issue Aug 12, 2024 · 0 comments
Labels
env:mamba For the mamba environment

Comments

@HaoZeke
Copy link
Member

HaoZeke commented Aug 12, 2024

For example:

asv run -j --launch-method spawn
· Creating environmentsTraceback (most recent call last):

  File "/home/rgoswami/micromamba/envs/asv_exp/bin/asv", line 8, in <module>
    sys.exit(main())
  File "/home/rgoswami/Git/Github/Quansight/asvWork/asv/asv/main.py", line 29, in main
    result = args.func(args)
  File "/home/rgoswami/Git/Github/Quansight/asvWork/asv/asv/commands/__init__.py", line 49, in run_from_args
    return cls.run_from_conf_args(conf, args)
  File "/home/rgoswami/Git/Github/Quansight/asvWork/asv/asv/commands/run.py", line 166, in run_from_conf_args
    return cls.run(
  File "/home/rgoswami/Git/Github/Quansight/asvWork/asv/asv/commands/run.py", line 278, in run
    Setup.perform_setup(environments, parallel=parallel)
  File "/home/rgoswami/Git/Github/Quansight/asvWork/asv/asv/commands/setup.py", line 74, in perform_setup
    pool.map(_create_parallel, environment_groups.values())
  File "/home/rgoswami/micromamba/envs/asv_exp/lib/python3.9/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/home/rgoswami/micromamba/envs/asv_exp/lib/python3.9/multiprocessing/pool.py", line 771, in get
    raise self._value
  File "/home/rgoswami/micromamba/envs/asv_exp/lib/python3.9/multiprocessing/pool.py", line 537, in _handle_tasks
    put(task)
  File "/home/rgoswami/micromamba/envs/asv_exp/lib/python3.9/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/home/rgoswami/micromamba/envs/asv_exp/lib/python3.9/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
TypeError: cannot pickle 'libmambapy.bindings.Context' object

This is an upstream issue, mamba-org/mamba#3403, though it remains to be seen if it will be supported.

@HaoZeke HaoZeke added plugin:mamba env:mamba For the mamba environment and removed plugin:mamba labels Aug 12, 2024
@HaoZeke HaoZeke changed the title BUG: Parallel execution needs serialization [mamba] BUG: Parallel execution needs serialization Aug 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
env:mamba For the mamba environment
Projects
None yet
Development

No branches or pull requests

1 participant