Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set PIP_USER in base image #437

Merged
merged 10 commits into from
Apr 17, 2024
Merged

Set PIP_USER in base image #437

merged 10 commits into from
Apr 17, 2024

Conversation

danielhollas
Copy link
Contributor

@danielhollas danielhollas commented Mar 19, 2024

Ensure that pip installs to ~/.local by default, instead of /opt/conda.

Couple other changes:

  • Do not pin pip version and always upgrade to latest, as recommended by pip maintainers. If there are any breaking changes in the future, they should be caught by the integration tests.
  • Install appmode from PyPI, we install it from Github only for historical reasons AFAIK.
  • Make --variant parameter of pytest required
  • Minor cleanup in tests
    Closes Set PIP_USER=true in the image #435

stack/base/Dockerfile Outdated Show resolved Hide resolved
@danielhollas
Copy link
Contributor Author

@unkcpz @mbercx it looks like docker daemon is not running on the ARM64 self-hosted runner.

https://github.com/aiidalab/aiidalab-docker-stack/actions/runs/8348422726/job/22850363649?pr=437

@danielhollas danielhollas requested a review from unkcpz March 19, 2024 18:54
@danielhollas
Copy link
Contributor Author

Arm runners are running again!

@mbercx
Copy link
Member

mbercx commented Mar 20, 2024

Arm runners are running again!

Yeah, sorry about that! Not sure why my work station keeps on shutting down from time to time. 🤷

@unkcpz
Copy link
Member

unkcpz commented Mar 20, 2024

Arm runners are running again!

Somehow there was an issue on the colima to start the aiidalab docker instance, and I delete the profile and restart. I think it bring the issue #432 to high priority.

@unkcpz
Copy link
Member

unkcpz commented Mar 20, 2024

Install appmode from PyPI, we install it from Github only for historical reasons AFAIK.

That's true.

ENV PIP_CONSTRAINT=/opt/requirements.txt
ENV PIP_CONSTRAINT /opt/requirements.txt
# Ensure that pip installs packages to ~/.local by default
ENV PIP_USER 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't get why this will make the pip install to ~/.local, can you point me to the pip documentation?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, sorry, this is actually equivalent to providing --user on the CLI.
You can set any pip argument via envvar by capitalizing it and pre-fixing with PIP_

unkcpz
unkcpz previously approved these changes Apr 8, 2024
Copy link
Member

@unkcpz unkcpz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @danielhollas Looks all good, one minor request on the comment.

docker-bake.hcl Show resolved Hide resolved
stack/lab/Dockerfile Outdated Show resolved Hide resolved
tests/conftest.py Show resolved Hide resolved
@danielhollas
Copy link
Contributor Author

danielhollas commented Apr 8, 2024

@unkcpz thanks for taking a look. I've pushed two new tests for the new pip behaviour. If you agree, I'll merge this and release a new latest version.

@danielhollas danielhollas enabled auto-merge (squash) April 8, 2024 14:41
@danielhollas danielhollas requested a review from unkcpz April 8, 2024 14:42
@danielhollas danielhollas mentioned this pull request Apr 8, 2024
@danielhollas
Copy link
Contributor Author

@unkcpz friendly ping 😊

@danielhollas danielhollas disabled auto-merge April 17, 2024 10:01
@danielhollas danielhollas merged commit 05d9932 into main Apr 17, 2024
25 checks passed
@danielhollas danielhollas deleted the pip-user branch April 17, 2024 10:04
@danielhollas
Copy link
Contributor Author

@unkcpz I've merged this, feel free to take a look at the new tests as part of reviewing #442

Copy link
Member

@unkcpz unkcpz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good, thanks for working on this @danielhollas
I had one question on the pip_install fixture.

Comment on lines +79 to +90
def pip_install(aiidalab_exec, nb_user):
"""Temporarily install package via pip"""
package = None

def _pip_install(pkg, **args):
nonlocal package
package = pkg
return aiidalab_exec(f"pip install {pkg}", **args)

yield _pip_install
if package:
aiidalab_exec(f"pip uninstall --yes {package}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember I mentioned somewhere there is an issue that the uninstallation will only remove the package but not the dependencies. Is this fixture solve that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, it doesn't. I am not sure if this is a problem in practice: Here I only use it for a package that I specifically chosen to have zero dependencies. But it would be good to at least document this.

I think it was a slightly bigger issue when running the integration tests and installing actuall AiiDAlab apps.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it was a slightly bigger issue when running the integration tests and installing actuall AiiDAlab apps.

You mean we should not test the QE-app or AWB in the full-stack image? That what I proposed but you said you were to comfortable with a image that not working with QeApp.
Can we make it a nightly test for QeApp and AWB that runs periodically?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it would be good to at least document this.

Yep, I'll add a comment on it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make it a nightly test for QeApp and AWB that runs periodically?

I don't think that is very developer friendly, I'd much rather now about issues rightaway.

I will note however that originally these app-installation tests were completely separate from the others and marked as "integration" tests, they specifically were not supposed to block merging. We should go back to that, I'll do that as part of #439, or as a separate PR after #442 is merged, which is a pre-requisite for this work.

danielhollas added a commit that referenced this pull request Apr 17, 2024
Ensure that pip installs to ~/.local by default, instead of /opt/conda.

Other minor changes:

* Do not pin pip version and always upgrade to latest, 
as recommended by pip maintainers. 
f there are any breaking changes in the future, 
they should be caught by the integration tests.
* Install appmode from PyPI
* Cleanup custom logo setup
* pytest: Make --variant a required parameter
* Simplify aiidalab_exec fixture usage
* Move test_pip_check to common tests
This was referenced Jul 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Set PIP_USER=true in the image
3 participants