Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add setup option for multiprocessed jobs with AlphaTwirl #55

Open
benkrikler opened this issue Jul 12, 2019 · 1 comment
Open

Add setup option for multiprocessed jobs with AlphaTwirl #55

benkrikler opened this issue Jul 12, 2019 · 1 comment

Comments

@benkrikler
Copy link
Member

This is particularly important when running from a conda installation and is effecting work on lxplus' htcondor.

Steps to implement things:

  1. Override alphatwirl's WorkingArea to be able to change the executable. The existing run.py script, the new wrapper script and the user's setup script must all be added to the workingArea tarballs
  2. Change the executable provided to htcondor's job description
  3. Create a simple executable wrapper that first runs the user's setup script, and then runs the normal run.py
  4. Add an option to fast-carpenter to point to the user's setup script.
@benkrikler benkrikler changed the title Add setup option for multiprocessed jobs Add setup option for multiprocessed jobs with AlphaTwirl Aug 7, 2020
@benkrikler
Copy link
Member Author

This issue is specific to the alphatwirl backend. It will require a bit of digging around in alphatwirl to implement. With the new Coffea backends I'm not sure how much effort we will be investing there in the future, but I will leave this issue open as a potential road map to implement this.

Note that we now have a mechanism to allow a user to provide this information from the command-line using the recently added execution config file added in PR #129.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant