-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathhow_to_install.qmd
125 lines (67 loc) · 7.66 KB
/
how_to_install.qmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
---
title: "How to Install"
---
The user interface allows to run and edit pipelines. It is used through the browser, but doesn't need to be installed on a web server. This can all happen locally. The code for the scripts, however, is not edited through the UI but rather with a standard text editor such as VS Code or RStudio.
## Video tutorial
{{< video https://www.youtube.com/watch?v=h3T36nLvg1E&t=4s >}}
## Deploying BON in a Box locally
BON in a Box can be installed and ran on a local computer. While personal computers have less computing power than servers, it is more convenient for pipeline development, or when working with small examples. Users adding scripts and creating pipelines on their local computer can directly run them through BON in a Box.
Prerequisites:
- A GitHub account, and git installed on your computer
- At least 6GB of free space (RAM requirements depend on the script that is being run)
- A linux shell
- This can be Terminal on a Mac or Git Bash on a PC
### Step 1: Install Docker
- Install Docker [here](https://www.docker.com/products/docker-desktop/)
- Note: It is not necessary to make an account
- To test whether docker is added to the path of your terminal, run `docker run --rm hello-world`
- **For Mac users:** Make sure to check the chip of your machine and download the correct version.
- If there is an error message when running `docker run --rm hello-world`, see <https://stackoverflow.com/a/71923962/3519951>
- If you encounter error `no matching manifest for linux/arm64/v8 in the manifest list entries`, export DOCKER_DEFAULT_PLATFORM. See <https://stackoverflow.com/a/76404045/3519951>.
- If you have a Silicon chip, check "Use Rosetta for x86_64/amd64 emulation on Apple Silicon". See [issue #175](https://github.com/GEO-BON/bon-in-a-box-pipelines/issues/175).
- **For linux users:** Make sure to [add your user to the docker group](https://docs.docker.com/engine/install/linux-postinstall/)
### Step 2: Generate SSH Key
1. First, make sure that you do not already have an SSH key registered to your computer by running `ls -al ~/.ssh` in your terminal.
- If you already have a key it will be called one of the following:
- id_rsa.pub
- id\_\_ecdsa.pub
- id_ed25519.pub
2. If you do not already have a key, generate one by running `ssh-keygen ed25519 -C "[email protected]"` replacing the email with the email address associated with your GitHub account.
- This will prompt you to make a password. We recommend skipping this because it becomes a hassle. You can keep pressing enter to skip these steps.
3. To check if you successfully generated the SSH key, you can run `ls -al ~/.ssh` in the terminal. Your SSH key should be there now.
4. Now, open the file with your SSH key by running `cat ~/.ssh/<name of SSH key>` e.g. `cat ~/.ssh/id_ed25519.pub` and copy your SSH key.
5. Now, add your SSH key to your GitHub account by going to GitHub settings \> SSH and GPG keys and "New SSH key". You can name this whatever you want and paste your SSH key into the "Key" field.
### Step 3: Clone repository
1. Clone repository (Windows users: do not clone the repo in a folder under OneDrive.) This can be done in terminal using the following code: `git clone [email protected]:GEO-BON/bon-in-a-box-pipelines.git` or in GitHub desktop.
### Step 4: Set environment file
1. Provide the environment variables: - Open the newly cloned repository on your computer
1. Find the file called [`runner-sample.env`](https://github.com/GEO-BON/bon-in-a-box-pipelines/blob/main/runner-sample.env)
2. Duplicate the file and rename the copy to `runner.env`.
3. Fill the properties depending on what you intend to run. Include any API keys that you need to access data (e.g. GBIF or IUCN)
4. Adjust any server option as you see fit.
2. Using a linux terminal (terminal on Mac or Git Bash), navigate to the folder of the cloned repository.
3. In the linux terminal, type `./server-up.sh`
- Make sure the server is started from a linux terminal that is in the same folder as the cloned repository so it can access the files.
- Make sure you have docker open and running on your computer.
- The first execution will be long, in order to download the micro-services. The next ones will be shorter or immediate, depending on the changes.
- Network problems may cause the process to fail. First try running the command again. Intermediate states are saved so not everything will be redone even when there is a failure.
- Windows users may need to turn on virtualization and other tools for Docker Desktop to work and update wsl ("wsl --update", see <https://docs.docker.com/desktop/troubleshoot/topics/#virtualization>. Access to the BIOS may be required to enable virtualization)
### Step 5: Connect to server
1. Type <http://localhost/> to open BON in a Box
2. Run `./server-down.sh` in a terminal to stop the server when done
3. On Windows, to completely stop the processes, you might have to run `wsl --shutdown`
When modifying scripts in the `/scripts` folder, servers do not need to be restarted: - When modifying an existing script, simply re-run the script from the UI and the new version will be executed. - When adding or renaming scripts, refresh the browser page.
When modifying pipelines in the /pipelines folder, servers do not need to be restarted: - In the pipeline editor, click save, paste the file to your file in the pipeline folder and run it from the "pipeline run" page. - When adding or renaming pipelines, refresh the browser page.
### Deploying without pulling the new server
If the server has already been deployed at least once, and low bandwith does not allow to download the latest version (this can be a few gigabytes), the server can be started with option `./server-up.sh --offline`. Use with caution, since this may create errors if elements of the repository depend on new server features.
## Deploying BON in a Box on a server {#deploying-the-servers-remotely}
Installation steps on a server are the same as installing on a Linux computer, but some additional configurations need to be changed:
- In `runner.env`, choose the appropriate `HTTP_ADDRESS` and `HTTP_PORT` for your server configuration. In a typical installation, use 0.0.0.0 as an HTTP address and default port (80).
- In `runner.env`, select the "partial" cache cleaning mode. This will allow for calculated results to be long-lived on the server, as long as they are not run again with a new version of the script. The results in the output folder are always accessible by URL.
- In the case where a server is used for demonstration purpose only, set environment variable `BLOCK_RUNS=true`
- The outputs are precious, while the server machine can be re-generated anytime. Consider mounting an external backed-up drive to the output folder of your BON in a Box installation.
- By default, BON in a Box listens for http. To enable https, we hide it behind a reverse proxy and activate https with certbot. How to do this is out of scope for this readme.
# Running a script or pipeline
You now have an instance of BON in a Box deployed, either [locally](#deploying-the-servers-locally) or [remotely](#deploying-the-servers-remotely), and you want to run your first script or pipeline.
There is one page to run scripts, and one to run pipelines. Once there, select the script or pipelines from the dropdown and fill the form.
The form might ask you for a file. In order to provide a file that you own locally, upload or copy it to the `userdata` folder. You can then refer to it with a url as such: `/userdata/myFile.shp`, or `/userdata/myFolder/myFile.shp` if there are subfolders.