Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deactivate Conda Env In HPC Init Script #388

Merged
merged 2 commits into from
Nov 8, 2024

Conversation

TimothyWillard
Copy link
Contributor

@TimothyWillard TimothyWillard commented Nov 7, 2024

Describe your changes.

This is to address a bug where a currently active env would cause the HPC init script to uncleanly load the flepiMoP env. Simplest solution is to always deactivate before activating. Redundant in the case that the flepiMoP env is already active, but easier than trying to determine the current env. No downside if no env is active.

What does your pull request address? Tag relevant issues.

No issue to tag, just noticed this issue while working on GH-365. Before this change (notice the flepimop-env is active):

(flepimop-env) [twillard@longleaf-login1 flepiMoP]$ source batch/hpc_init.sh longleaf
An explicit $FLEPI_PATH was not provided, please set one (or press enter to use '/work/users/t/w/twillard/flepiMoP'):
Using '/work/users/t/w/twillard/flepiMoP' for $FLEPI_PATH.
An explicit $FLEPI_CONDA was not provided, please set one (or press enter to use 'flepimop-env'):
Using 'flepimop-env' for $FLEPI_CONDA.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'pyarrow'
Connection to longleaf.unc.edu closed.

and after:

(flepimop-env) [twillard@longleaf-login6 flepiMoP]$ source batch/hpc_init.sh longleaf
An explicit $FLEPI_PATH was not provided, please set one (or press enter to use '/work/users/t/w/twillard/flepiMoP'):
Using '/work/users/t/w/twillard/flepiMoP' for $FLEPI_PATH.
An explicit $FLEPI_CONDA was not provided, please set one (or press enter to use 'flepimop-env'):
Using 'flepimop-env' for $FLEPI_CONDA.
Please set a project path (relative to '/work/users/t/w/twillard'):
...

Tag relevant team members.

@anjalika-nande, @emprzy, @fang19911030, @jcblemai, @pearsonca, @saraloo

@TimothyWillard TimothyWillard added bug Defects or errors in the code. batch Relating to batch processing. quick issue Short or easy fix. labels Nov 7, 2024
Copy link
Contributor

@pearsonca pearsonca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is likely fine.

I would prefer to be a bit more user-friendly, by issuing some kind of warning maybe (probably only if environment name != flepimop-env)? Like some stderr text along the lines of "Detected an active conda environment $X; this will be deactivated and the flepimop-env will be activated".

I don't think it needs to be a "proceed [Y/n]" situation, but my concern here is that we're introducing a subtle side-effect.

Marking this as approved - open to addressing the above in this PR (either fix or convincing me wontfix) or as own issue.

@TimothyWillard TimothyWillard force-pushed the fix-activate-conda-breaking-hpc-init branch from 1e8cbd8 to 1c81583 Compare November 7, 2024 15:39
@TimothyWillard
Copy link
Contributor Author

@pearsonca I've incorporated a message now if the active environment does not match the one being activated. The new after is:

(test-env) [twillard@longleaf-login6 flepiMoP]$ source batch/hpc_init.sh longleaf
An explicit $FLEPI_PATH was not provided, please set one (or press enter to use '/work/users/t/w/twillard/flepiMoP'):
Using '/work/users/t/w/twillard/flepiMoP' for $FLEPI_PATH.
An explicit $FLEPI_CONDA was not provided, please set one (or press enter to use 'flepimop-env'):
Using 'flepimop-env' for $FLEPI_CONDA.
Detected an active conda environment 'test-env'. This will be deactivated and the 'flepimop-env' environment will be activated.
Please set a project path (relative to '/work/users/t/w/twillard'):
...

This is to address a bug where a currently active env would cause the
hpc init script to load the flepimop env not cleanly. Determine the
currently active env if there is one and if it is not the flepimop env
deactivate it with message to the user.
@pearsonca
Copy link
Contributor

looks good to merge to me; @TimothyWillard feel free to trigger unless you have some additional tweaks in mind.

@TimothyWillard TimothyWillard force-pushed the fix-activate-conda-breaking-hpc-init branch from 1c81583 to d4c3c7f Compare November 7, 2024 15:53
@TimothyWillard
Copy link
Contributor Author

looks good to merge to me; @TimothyWillard feel free to trigger unless you have some additional tweaks in mind.

I'm still testing/modifying the changes requested. I shouldn't have requested a re-review yet and this is not ready to merge.

1) No env active, just activate the flepiMoP env.
2) The flepiMoP env is active, go ahead and refresh it because
   `module load anaconda` modifies the paths.
3) Another conda env is active, deactivate it with a message.
@TimothyWillard
Copy link
Contributor Author

I apologize for the confusion. This is now tested and ready for review. The three cases look like:

  1. When a different conda env is already loaded:
[twillard@longleaf-login4 flepiMoP]$ conda activate test-env
(test-env) [twillard@longleaf-login4 flepiMoP]$ source batch/hpc_init.sh longleaf
An explicit $FLEPI_PATH was not provided, please set one (or press enter to use '/work/users/t/w/twillard/flepiMoP'):
Using '/work/users/t/w/twillard/flepiMoP' for $FLEPI_PATH.
An explicit $FLEPI_CONDA was not provided, please set one (or press enter to use 'flepimop-env'):
Using 'flepimop-env' for $FLEPI_CONDA.
Detected an active conda environment 'test-env'. This will be deactivated and the 'flepimop-env' environment wil be activated.
Please set a project path (relative to '/work/users/t/w/twillard'): ^C
  1. When the flepiMoP conda env is already loaded (flepimop-env in my case):
(flepimop-env) [twillard@longleaf-login4 flepiMoP]$ source batch/hpc_init.sh longleaf
An explicit $FLEPI_PATH was not provided, please set one (or press enter to use '/work/users/t/w/twillard/flepiMoP'):
Using '/work/users/t/w/twillard/flepiMoP' for $FLEPI_PATH.
An explicit $FLEPI_CONDA was not provided, please set one (or press enter to use 'flepimop-env'):
Using 'flepimop-env' for $FLEPI_CONDA.
Detected the activate conda environment is 'flepimop-env' already, but will refresh.
Please set a project path (relative to '/work/users/t/w/twillard'): ^C
  1. Probably the most common case, when no conda env is loaded:
[twillard@longleaf-login4 flepiMoP]$ source batch/hpc_init.sh longleaf
An explicit $FLEPI_PATH was not provided, please set one (or press enter to use '/work/users/t/w/twillard/flepiMoP'):
Using '/work/users/t/w/twillard/flepiMoP' for $FLEPI_PATH.
An explicit $FLEPI_CONDA was not provided, please set one (or press enter to use 'flepimop-env'):
Using 'flepimop-env' for $FLEPI_CONDA.
Please set a project path (relative to '/work/users/t/w/twillard'): ^C

As always feedback from ops is greatly appreciated, especially since this script is designed for ops use.

@pearsonca pearsonca self-requested a review November 7, 2024 16:55
Copy link
Collaborator

@anjalika-nande anjalika-nande left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is very clear!

@saraloo
Copy link
Contributor

saraloo commented Nov 8, 2024

Thanks, this is great :)

@TimothyWillard TimothyWillard merged commit b07b9f6 into dev Nov 8, 2024
@TimothyWillard TimothyWillard deleted the fix-activate-conda-breaking-hpc-init branch November 8, 2024 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
batch Relating to batch processing. bug Defects or errors in the code. quick issue Short or easy fix.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants