installation.qmd

---
title: Installation Instructions
engine: knitr
---

```{r echo=FALSE}
knitr::opts_chunk$set(
  echo = TRUE,
  comment = "",
  collapse = FALSE,
  prompt = FALSE
)

local({
  hook_old <- knitr::knit_hooks$get("output")  # save the old hook
  knitr::knit_hooks$set(output = function(x, options) {
    regex = "(?:(?:\\x{001b}\\[)|\\x{009b})(?:(?:[0-9]{1,3})?(?:(?:;[0-9]{0,3})*)?[A-M|f-m])|\\x{001b}[A-M]"
    out <- gsub(regex, "", x, perl = TRUE, useBytes = TRUE)
    hook_old(out, options)
  })
})

```

## Introduction

LuciusOperations uses [Viash](https://viash.io) to convert relatively simple scripts containing REST calls into configurable CLI tools.

The use of Viash and other applications of the tool is outside the scope of this document. Suffice to note that Viash uses a concept of _source_ and what is built from that source. The source of the components can be found under `src/`, using the `bin/build.sh` wrapper script, the built scripts are under `utils/`.

```{sh}
bin/build.sh
```

::: {.callout-note}
This description deals with the installation and configuration of LuciusOperations only.
:::

## Where to run

### Generic installation

It does not matter where the tools in this toolset are used, as long as `wget` and `curl` are available on the device and the DNS endpoints of the Spark Jobserver can be resolved, it should work.

In order to run the processing jobs in `api` and `processing`, however, it's important to collect the appropriate JAR files. Those define the logic (i.e. the code) to actually run on the Spark cluster. Both the `api` as well as the `processing` toolset have an `upload_jar` tool that can be used to upload the appropriate JAR file to the Spark Jobserver.

We will often run the LuciusOperations tools from the SparkJobser instance itself. This allows us to use the (locally) available JAR files that have been used to initialize the API in the first place. Connecting to a Spark Jobserver instance depends on your installation.

### Suggested installation

We suggest to run the LuciusOperations tools from an instance running the Lucius backend. Nothing has to be installed in that case, please see [here](usage.qmd#suggested-setup) for more information.

## Technical details

Every Viash component contains a `script.sh` file (usually some parameter handling and a command to execute) and a `config.vsh.yaml` file which contains the configuration for the component.

By running `bin/build.sh`, this combination of files is transformed into 1 executable script that performs essentially two things:

1. The resulting executable contains the argument parsing necessary to run it from the CLI
2. The defaults configured in `_viash.yaml` are applied.

An illustration of the CLI argument parsing capabilities:

```{sh}
utils/processing/check -h | head
```

The format of the configuration in `_viash.yaml` is derived from the way [jq](https://stedolan.github.io/jq/) allows to query and update JSON blobs. It should be clear from the example in the repository how to use it:

```{embed lang="yaml"}
_viash.yaml
```

The `bin/build.sh` script uses Viash to create 2 subdirectories under `utils/` containing tools that are used for the _processing_ part and for the _api_ part:

```{sh}
tree utils/
```

Please refer to the [usage](usage.html) guide for more information about how to use the tools under `utils/`.