Merge pull request #632 from douglasjacobsen/requirements-dev

Add a requirements-dev to separate development and user requirements

README.md

@@ -34,9 +34,14 @@ support Python 3.6 as it is used by older operating systems such as Centos7.
 Specifically, you might need to update `pip` and downgrade `protobuf` when
 using Python 3.6.
 
-Outside of these requirements, ramble requires an existing installation of
-spack for some application definition. See
-[Spack’s documentation](https://github.com/spack/spack#-spack) to install Spack.
+Outside of these requirements, Ramble requires package managers to be
+configured if they will be used as part of the experiments Ramble creates.
+
+Although package manager support is not required to use Ramble, some
+experiments are more easily accomplished by allowing Ramble to drive the
+package manager execution. To allow this, package managers generally need to be
+installed indepednently from Ramble. For more information on this, see
+[Ramble's supported package managers](https://ramble.readthedocs.io/en/latest/package_managers.html).
 
 Documentation
 ----------------
@@ -65,6 +70,14 @@ Resources:
 
 Contributing
 ------------------------
+When developing features for Ramble, it can be helpful to install the
+development requirements instead of the user requirements:
+
+e.g.
+
+    pip install -r requirements-dev.txt
+
+
 Contributing to Ramble is relatively easy.  Just send us a
 [pull request](https://help.github.com/articles/using-pull-requests/).
 When you send your request, make ``develop`` the destination branch on the

lib/ramble/docs/package_managers.rst

@@ -75,6 +75,11 @@ into each experiment, to prepare the execution environment.
 The :ref:`software configuration section <software-config>` is used to
 determine what modules to load within the resulting environment.
 
+The use of this package manager requires an installation of
+``environment-modules`` outside of Ramble. This is handled by system
+administrators on several clusters, but for more information see
+`environment-modules's documentation <https://modules.readthedocs.io/en/latest/INSTALL.html>`.
+
 ^^^^^^^^^^^^^^^^^^^^^
 EESSI Package Manager
 ^^^^^^^^^^^^^^^^^^^^^
@@ -87,6 +92,10 @@ EESSI, and load the correct module files for the execution environment.
 The :ref:`software configuration section <software-config>` is used to
 determine what modules to load within the resulting environment.
 
+The use of this package manager requires an installation of EESSI outside of
+Ramble. For more information, see
+`EESSI's documentation <https://www.eessi.io/docs/getting_access/native_installation/>`.
+
 ^^^^^^^^^^^^^^^^^^^
 Pip Package Manager
 ^^^^^^^^^^^^^^^^^^^
@@ -101,6 +110,11 @@ manager.
 The :ref:`software configuration section <software-config>` is used to
 determine what packages to install within the resulting virtual environment.
 
+The use of this package manager requires ``pip`` to be installed outside of
+Ramble. This happens automatically in several Python installations. For more
+information see
+`pip's documentation<https://pip.pypa.io/en/stable/installation/>`.
+
 ^^^^^^^^^^^^^^^^^^^^^
 Spack Package Manager
 ^^^^^^^^^^^^^^^^^^^^^
@@ -118,6 +132,10 @@ When using the Spack package manager, workspaces can also use the
 ``push-to-cache`` and ``mirror`` pipelines to cache compiled binaies, and
 mirror software source.
 
+The use of this package manager requires an external installation of Spack. For
+instructions on installing Spack, see
+`Spack's documentation <https://github.com/spack/spack#-spack>`.
+
 
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 Spack Lightweight Package Manager
@@ -137,3 +155,6 @@ When using the Spack package manager, workspaces can also use the
 ``push-to-cache`` and ``mirror`` pipelines to cache compiled binaies, and
 mirror software source.
 
+The use of this package manager requires an external installation of Spack. For
+instructions on installing Spack, see
+`Spack's documentation <https://github.com/spack/spack#-spack>`.

requirements-dev.txt

@@ -0,0 +1,6 @@
+-r requirements.txt
+pytest
+flake8
+black
+coverage
+pre-commit

requirements.txt

@@ -1,10 +1,5 @@
-pytest
-flake8
-black
 google-cloud-storage # for gcs fetch test
 google-api-core # for gcs fetch error .execptions
-coverage
-pre-commit
 graphlib-backport;python_version<"3.9"
 urllib3==1.26.18;python_version<="3.6"
 protobuf;python_version>"3.6"

share/ramble/cloud-build/Dockerfile-apt

@@ -17,7 +17,8 @@ RUN cd /opt &&  \
     . spack/share/spack/setup-env.sh && \
     spack load py-pip ^python@${PYTHON_VER} && \
     wget https://raw.githubusercontent.com/GoogleCloudPlatform/ramble/develop/requirements.txt && \
-    python -m pip install -r /opt/requirements.txt
+    wget https://raw.githubusercontent.com/GoogleCloudPlatform/ramble/develop/requirements-dev.txt && \
+    python -m pip install -r /opt/requirements-dev.txt
 
 FROM ${BASE_IMG}:${BASE_VER}
 

share/ramble/cloud-build/Dockerfile-yum

@@ -17,7 +17,8 @@ RUN cd /opt &&  \
     . spack/share/spack/setup-env.sh && \
     spack load py-pip ^python@${PYTHON_VER} && \
     wget https://raw.githubusercontent.com/GoogleCloudPlatform/ramble/develop/requirements.txt && \
-    python -m pip install -r /opt/requirements.txt
+    wget https://raw.githubusercontent.com/GoogleCloudPlatform/ramble/develop/requirements-dev.txt && \
+    python -m pip install -r /opt/requirements-dev.txt
 
 FROM ${BASE_IMG}:${BASE_VER}