Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

uproot4 error with coffea:local backend #136

Open
asnaylor opened this issue Oct 20, 2020 · 5 comments
Open

uproot4 error with coffea:local backend #136

asnaylor opened this issue Oct 20, 2020 · 5 comments

Comments

@asnaylor
Copy link
Collaborator

asnaylor commented Oct 20, 2020

I tried to use the coffea local backend which has worked for me previously:

$ fast_carpenter dataset.yml local_cfg.yml --no-bookkeeping --mode coffea:local

But today I got this error message about uproot4 not having MultithreadedFileSource.

Trace:
Preprocessing: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:02<00:00,  2.00s/file]
Processing:   0%|                                                                                                                                       | 0/1 [00:00<?, ?chunk/s]
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/concurrent/futures/process.py", line 239, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/site-packages/coffea/processor/executor.py", line 129, in __call__
    out = self.function(*args, **kwargs)
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/site-packages/coffea/processor/executor.py", line 872, in _work_function
    raise e
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/site-packages/coffea/processor/executor.py", line 790, in _work_function
    file_handler=uproot4.MemmapSource if mmap else uproot4.MultithreadedFileSource,
AttributeError: module 'uproot4' has no attribute 'MultithreadedFileSource'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/bin/fast_carpenter", line 8, in <module>
    sys.exit(main())
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/site-packages/fast_carpenter/__main__.py", line 72, in main
    results, _ = backend.execute(sequence, datasets, args)
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/site-packages/fast_carpenter/backends/coffea.py", line 175, in execute
    out = cop.run_uproot_job(coffea_datasets, 'events', fp,
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/site-packages/coffea/processor/executor.py", line 1123, in run_uproot_job
    executor(chunks, closure, wrapped_out, **exe_args)
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/site-packages/coffea/processor/executor.py", line 557, in futures_executor
    _futures_handler(futures, accumulator, status, unit, desc, add_fn, tailtimeout)
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/site-packages/coffea/processor/executor.py", line 187, in _futures_handler
    add_fn(output, finished.pop().result())
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
AttributeError: module 'uproot4' has no attribute 'MultithreadedFileSource'

Versions:

python_version=3.8.5
fast-carpenter==0.19.0
coffea==0.6.45
@asnaylor
Copy link
Collaborator Author

asnaylor commented Oct 20, 2020

The coffea team introduced uproot4 in version 0.6.43 so I tested the above command with 0.6.42 and it worked fine. Until they have a fix, you can just pin fast_carpenter to use coffea==0.6.42.

@kreczko
Copy link
Contributor

kreczko commented Dec 4, 2020

uproot 4.0.0rc2 is now available, together with awkward 1.0.0rc2.

I've now released scikit-validate 0.4.0rc1 against the two above, so I can start looking at this here next.

@benkrikler What are your plans w.r.t. the new uproot and awkward versions?
They have now moved into a stable territory, so we can try to move.
BTW: This might certainly spell the end of Python 2 support (FINALLY!)

@benkrikler
Copy link
Member

I want to move over to that as soon as possible, and fully migrate from python 2 at the same time (we're already no longer testing support there, but I think we might still have some six things and from __future__ stuff going on). If you want to tackle moving us to uproot 4 already then that would be great!

@kreczko
Copy link
Contributor

kreczko commented Nov 25, 2021

The current main issue with coffea: There is no simple way anymore to extract the tree.

I will try today to use the df directly, but I need to check other code dependencies first.

@kreczko
Copy link
Contributor

kreczko commented Mar 28, 2022

This will be fixed in #144

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants