Merge pull request #2 from PlasmaFAIR/viking_upgrades

Viking2 upgrades

Dockerfile

@@ -29,7 +29,7 @@ RUN git clone --recursive https://github.com/Warwick-Plasma/epoch /app/epoch
 
 # Set up Python, install utility library
 RUN python3 -m pip install --upgrade pip
-RUN python3 -m pip install numpy matplotlib
+RUN python3 -m pip install numpy scipy matplotlib pandas xarray
 RUN python3 -m pip install .
 
 # Build Epoch variants

README.md

@@ -1,7 +1,7 @@
 # Epoch Containers
 
-Tools and information for building/running [Epoch][epoch] using Docker/Singularity
-containers.
+This repository contains tools and information for building and running [Epoch][epoch]
+using containers.
 
 ## Introduction
 
@@ -11,127 +11,64 @@ allow researchers to run code without needing to build it themselves, and they m
 it much easier to share reproducible workflows.
 
 We provide support for two container platforms: [Docker][docker] and
-[Singularity][singularity]. Docker is the most widely used platform, and has been used
-here to build a 'base image' of Epoch on which other tools may be built. Singularity is
-an alternative container platform that was designed from the ground up to be useable on
-HPC systems, so unlike Docker it can be run on multi-node architectures using MPI
-without issue.
+[Singularity][singularity] (or Apptainer). Docker is the most widely used platform, and
+has been used here to build a 'base image' of Epoch on which other tools may be created.
+Singularity is an alternative container platform that is better suited for use on HPC
+systems, as unlike Docker it can be run using MPI.
 
 ## Usage
 
 Users of this software do not need to build containers themselves. Instead, they will
 only need to copy the Python module `run_epoch.py` in the root directory of this
-repository. To run the Docker container, try:
-
-```python
-$ python3 run_epoch.py docker -d 2 -o ./my_epoch_run
-```
-
-Here, `-d` specifies the number of dimensions to run (e.g. here we are performing a 2D
-simulation), `-o` specifies the output directory (which should contain `input.deck`).
-Users can switch on QED effects by also providing the `--photons` argument. The output
-directory should not be the current working directory.
-
-Similarly, to run the Singularity container, try:
-
-```python
-$ python3 run_epoch.py singularity -d 2 -o ./my_epoch_run -n 4
-```
-
-The extra argument `-n` specifies the number of processes to run, which uses OpenMPI.
-On HPC systems, you will need to load Singularity and OpenMPI first. For example, on
-Viking at the University of York, this requires:
-
-```bash
-$ module load tools/Singularity mpi/OpenMPI
-```
-
-## Running on Viking (University of York)
-
-To run Epoch on Viking, first create a directory within `~/scratch` in which you
-want to run your code:
-
-```
-$ ssh <userid>@viking.york.ac.uk
-$ mkdir -p ~/scratch/epoch/output
-$ cd ~/scratch/epoch
-```
-
-You'll need to ensure your `input.deck` file is within this directory:
-
-```bash
-$ # From your own machine
-$ scp input.deck <userid>@viking.york.ac.uk:/users/<userid>/scratch/epoch/output
-```
-
-To run the Singularity container, you'll need to load the following modules:
+repository. If we run this with the argument `--help`, we can see a list of possible
+commands:
 
 ```bash
-$ module load tools/Singularity mpi/OpenMPI
+$ python3 run_epoch.py --help
 ```
 
-You may then run the helper script as described above.
+### Docker
 
-Note that you should only run short tests on the login nodes. To run longer jobs, you'll
-want to create a Slurm job file. See the `./examples` folder for an example job script
-`run_sbatch.sh` and an example `input.deck`. Once we have a job script, we can submit a
-job using:
+We can run the Docker container using:
 
 ```bash
-$ sbatch run_sbatch.sh
+$ python3 run_epoch.py docker -d 2 -o ./my_epoch_run
 ```
 
-We can check the progress of our job using:
+Here, `-d` specifies the number of dimensions in the simulation and `-o` specifies the
+output directory (which should contain `input.deck`). Users can switch on QED effects by
+also providing the `--photons` argument. The output directory should not be the current
+working directory. To see a full list of possible options, we can also supply the
+`--help` option:
 
 ```bash
-$ squeue -u <userid>
+$ python3 run_epoch.py docker --help
 ```
 
-## Inspecting the Container
-
-It is also possible to pull the container from the remote repo:
-
-```bash
-$ singularity pull epoch.sif oras://ghcr.io/plasmafair/epoch.sif:latest
-```
+### Singularity/Apptainer
 
-This will download the container image to the file `epoch.sif` (`.sif` denoting a
-'Singularity Image Format' file). You can then use `epoch.sif` in place of
-`library://account/repo/container` in any of the commands above.
+Singularity containers can be run similarly to Docker containers, but with extra options
+for specifying the number of MPI processes to run:
 
-If you want to inspect the container, we can use:
-
-```bash
-$ singularity shell epoch.sif
+```python
+$ python3 run_epoch.py singularity -d 2 -o ./my_epoch_run -n 4
 ```
 
-## Analysing code output
+The extra argument `-n` specifies the number of processes to run. With Singularity, it
+is permitted to run in the current working directory.
 
-It is recommended to analyse Epoch output data on your own machine rather than on an HPC
-machine:
+On HPC systems, you will need to load OpenMPI and Singularity/Apptainer first. For
+example, on Viking at the University of York, this requires:
 
 ```bash
-$ scp <userid>@viking.york.ac.uk:/users/<userid>/scratch/epoch/*.sdf .
+$ module load OpenMPI Apptainer
 ```
 
-You'll need a particular Python library to read `.sdf` files, and this is packaged with
-Epoch itself. To install this library, try:
-
-```bash
-$ git clone https://github.com/Warwick-Plasma/epoch
-$ cd epoch/epoch1d
-$ make sdfutils
-```
-
-Note that the SDF Python library is not packaged with modern best-practices in mind
-(i.e. using virtual environments, uploading packages to PyPI/conda-forge). It will
-install to `~/.local/lib/python3.x/site-packages` regardless of whether you're in a
-`venv` or `conda` environment. If you feel you know what you're doing, you can manually
-copy/move the installed files to the environment of your choice after installing, but
-it's recommended to just use the base user environment.
-
-Please see the [Epoch docs][epoch] for info on using SDF analysis tools.
+Some machines may need to load a specific version of OpenMPI -- the version in the
+container is 4.1.2.
 
+Please see the `./viking` directory for help with running on Viking. This also contains
+advice for processing the SDF files produced by Epoch.
 
 ## Licensing