Skip to content

Commit

Permalink
Revert "Update readme for setting up Docker images"
Browse files Browse the repository at this point in the history
This reverts commit 4e7479c.
  • Loading branch information
sf1919 committed Apr 25, 2023
1 parent 61db683 commit 59b0858
Show file tree
Hide file tree
Showing 2 changed files with 98 additions and 150 deletions.
127 changes: 98 additions & 29 deletions Linux/development/readme.md
Original file line number Diff line number Diff line change
@@ -1,52 +1,121 @@
# Mantid development environment in Docker for Linux
# Mantid development environment in Docker

Docker container for building, testing and developing Mantid in Linux. If your host system is Windows then it might be better to setup a [Windows Subsystem for Linux (WSL2)](https://developer.mantidproject.org/WindowsSubsystemForLinux.html).
Docker container for building, testing and developing Mantid. If your host system is Windows then it might be better to setup a [Windows Subsystem for Linux (WSL2)](https://developer.mantidproject.org/WindowsSubsystemForLinux.html).

## Base images and versions

There is only one base image:
There are images for each base OS:

- [`mantid-development-centos7-slim`](https://github.com/mantidproject/dockerfiles/pkgs/container/mantid-development-centos7-slim/) - CentOS 7

CentOS 7 is the OS used by conda forge.
- [`mantidproject/mantid-development-centos7`](https://hub.docker.com/r/mantidproject/mantid-development-centos7/) - CentOS 7
- [`mantidproject/mantid-development-ubuntubionic`](https://hub.docker.com/r/mantidproject/mantid-development-ubuntubionic/) - Ubuntu 18.04 (Bionic)

Typically you'd want to use the `latest` tag, this corresponds to the latest version of the developer package.

## Jenkins image
If you do need a specific developer package version then they are available as tags.
Tags follow the naming convention of `devpkg-VER`, where `VER` is the developer package version installed.

## Usage

A separate image is available for the Jenkins set up of a Linux node:
The images contain three directories `/mantid_src`, `/mantid_build` and `/mantid_data` which are to be used for the source, build and CMake external data directories respectively.
It is recommended to have these directories mounted to locations on the host filesystem. Reasons being:

- [`jenkins-node-centos7-slim`](https://github.com/mantidproject/dockerfiles/pkgs/container/jenkins-node-centos7-slim/)
- Using your existing SCM, editors, etc. (you modify the code on the host filesystem as you will have probably already been doing)
- Using common external data for host builds and all container builds
- Running a container built or packaged Mantid on the host (assuming an appropriate host system and base image)

As with the base image you typically want to use the `latest` tag.
To ensure file permissions are handled correctly if mapping volumes to your host filesystem you must pass the `PUID` and `PGID` environment variables when starting a container, these should be set to your user ID and group ID respectively.

The `mantid_development.sh` script can be used to start a container, this script takes four parameters:
```sh
./mantid_development.sh [os] [source] [build] [external data]
```

## Updating images
`[os]` is the image variant you want to use (one of `centos7` or `ubuntubionic`).

For changes to the main image update the [`CentOS7Slim.Dockerfile`](https://github.com/mantidproject/dockerfiles/tree/main/Linux/development/docker). You will also need to update the `build_common_slim.sh` file in the same folder with an updated version number.
`[source]`, `[build]` and `[external data]` are the volumes which will be mounted as the source (root of the Mantid Git repository), build and CMake external data directories.
These can either be paths to the host filesystem or names of Docker volumes.

If you make changes to the main file you will also need to update the Jenkins image as this specifies which version of the base image you want to use. This file is also called [`CentOS7Slim.Dockerfile`](https://github.com/mantidproject/dockerfiles/tree/main/Linux/jenkins-node/docker) but is located within the jenkins-node subfolders. As with the base image you will also need to update the `build_common_slim.sh` file in the same folder with an updated version number.
This will give you a `bash` shell in the build directory.
From here you can run `cmake` and your build tool of choice just as you would on your host OS.
Inside the container you will have the username `abc` which is a standard (i.e. non-root) user with `sudo` ability.

## Building locally
The `mantid_development.sh` (and `mantid_development_x11docker.sh`) scripts may need to be modified to suit your system and the environment that you are running them under.
In their current state they are a reasonable default.

On Linux an easy way to build a docker container from the image is to run the `build_slim.sh` file from the same folder as the image you want to build.
All images contain a script (`$HOME/configure.sh`) which will perform a sensible CMake configuration ready for building.
Of course, this can be done manually if a specific configuration is required, however the script should be inspected to find common paths, etc.

### GUI

For running GUI parts of Mantid (i.e. MantidPlot and workbench) the easiest option is to use [`x11docker`](https://github.com/mviereck/x11docker) via `mantid_development_x11docker.sh`:
```sh
./bin/mantid_development_x11docker.sh [os] [source] [build] [external data] [cmd]
```

## Publishing updated images
If you don't want to or can't use `x11docker` then you can try using simple X server mapping (see section below).

All our images are hosted on GitHub instead of Docker. In order to publish new images to GitHub you will need to do the following
### Network proxy

- run the `build_slim.sh` file to ensure the new image builds without issue
- generate a new GitHub token that will give you read and write access to Mantid project
- Ensure you have your conda environment active and that docker is available (run the command ```docker run hello-world``` to check this)
- set up your GitHub token using the following as a guide. Be sure to use your own GitHub user name.
One way to get networking to work over a proxy server is to directly use the host system's networking from Docker.
First, one needs to enable port forwarding.
On Ubuntu 16.04 this can be done by
```sh
sudo sysctl net.ipv4.conf.all.forwarding=1
sudo iptables -P FORWARD ACCEPT
```
GH_TOKEN=XXXXXXXXXXXXXXXX
echo "$GH_TOKEN" | docker login ghcr.io -u YourGitHubUserName --password-stdin

Next, the container has to be launched with the `--network host` option in `docker run` command.
To actually specify the proxy settings, pass `--env http_proxy="http://proxy.domain.tv:2323"` and `--env https_proxy="https://proxy.domain.tv:5555"` to the command.

### Advanced/non-standard usage

The `mantid_development.sh` and `mantid_development_x11docker.sh` scripts are the bare minimum requirements for developing under Docker.
This section gives some examples of additional arguments you may want to include in those scripts for specific purposes.

All examples are arguments to `docker run`.
If you'd like to use any with `x11docker` you must pass them in the manner shown in the `mantid_development_x11docker.sh`.

#### Debugging

Removes security restrictions that would usually prevent a debugger from running correctly.
Tested with GDB.

```sh
--security-opt seccomp=unconfined
--cap-add=SYS_PTRACE
```
- run the `push_slim.sh` file
- Check the page for the image pushed to check that it has uploaded as expected. This page can be accessed from the [`Mantid Packages page`](https://github.com/orgs/mantidproject/packages)
- If you have pushed the base image you will also need to push the Jenkins image
- Once you are happy that the new image works as expected delete any older versions. At any one time we will only have the current and one previous image available.

## Using images
See [`Ansible readme`](https://github.com/mantidproject/dockerfiles/blob/main/Linux/jenkins-node/ansible/readme.md) for details of using ansible scripts to set up nodes using these Linux docker images.
#### X server access

Adds X server forwarding for "standard" Docker use (i.e. not `x11docker`).
Host IPC namespacing is required for some Qt functionality.

```sh
--env DISPLAY="$DISPLAY"
--volume "/tmp/.X11-unix:/tmp/.X11-unix:ro"
--volume "$HOME/.Xauthority:/home/abc/.Xauthority:ro"
--ipc=host
```

#### Network access

Some network functionality (e.g. live streaming) may require (or at least become easier with) host network access.

```sh
--net=host
```

#### Persistent home directory

Useful for disabling the painfully annoying "Welcome to Mantid!" splash screen.
Also persisting IDF updates, shell history and such.

```sh
--volume mantid_development_home:/home/abc
```

#### Providing access to data

```sh
man docker-run
```
121 changes: 0 additions & 121 deletions development/readme.md

This file was deleted.

0 comments on commit 59b0858

Please sign in to comment.