Composed Workspace

Quick setup & run

1. Install Docker Desktop: https://docs.docker.com/engine/install/

2. Login your Github account:

   Create a personal access token (classic, not fine-grained) with the scope read:packages checked. You can do this from your Github account: Creating a personal access token.

   Once the token is generated, within the project directory run the following command, replacing <GITHUB_USER> with your Github username and <TOKEN> with the generated one:

   ./wspace login <GITHUB_USER> <TOKEN>

3. Start the Composed Workspace deploy running:

   ./wspace init

   The whole process will take around 12 minutes and 2.3 GiB of downloaded data, so it's recomended to have a cup of coffee while waiting. Once the downloads, builds, migrations and deployments are done for the first time, the consecutive runs will last less than 1 minute with the ./wspace up command.

4. Once the Composed Workspace is deployed, to see usage info navigate to: http://localhost:4010

5. Finally, to stop the Composed Workspace run:

   ./wspace stop

Table of Contents

• Composed Workspace
  • Quick setup & run
  • About
    • Architecture diagram
    • Workspace services
    • Workspace tools
    • Valiot App services
  • Script usage info
  • System setup
    • Authentication
  • Workspace setup
    • Initialization
    • Building containers
    • Databases and services setup
  • Workspace deploy
    • All services
    • Single service
  • Stop workspace
    • Stop service(s)
    • Remove services
    • Docker reset
  • Services' CLI terminal
  • Change UI & Worker services
    • Docker images
    • Localhost services
      • Localhost UI
      • Localhost Worker
  • Workspace for development
    • Valiot App
      • Start Valiot App services
      • Build Valiot App containers
      • Stop Valiot App services
      • Valiot App tasks

About

This workspace is an all-in-one solution for effortlessly deploy, test and develop the Valiot services ecosystem in a local machine without the need of install anything else than Docker Desktop.

It's built over Docker containers that resemble the context architecture. Therefore, it guarantees that the deployment behavior will be the same in prod environment.

Architecture diagram

Workspace services

Service name	Image version
ui	ghcr.io/valiot/valiot-ui:ui-demo-8961af5
exp-627	ghcr.io/valiot/exp-627:1.4.0
worker	ghcr.io/valiot/vcos-test-worker:e0045fc
alerts	ghcr.io/valiot/alerts:5.0.0
auth	ghcr.io/valiot/valiot-auth:3.3.0
blog	ghcr.io/valiot/blog:2.1.0
eliot	ghcr.io/valiot/eliot:8.1.0
jobs	ghcr.io/valiot/jobs:6.0.0
notifications	ghcr.io/valiot/notifications:5.0.0
schedule-logic	ghcr.io/valiot/schedule-logic:4.0.0
ui-config	ghcr.io/valiot/ui-config:2.0.0
user	ghcr.io/valiot/valiot-user:4.3.0
elixir-apps-db	postgres:14

Workspace tools

Service name	Image version
wspace-ui	Source code v1.5.0
pgadmin	dpage/pgadmin4:latest

Valiot App services

Service name	Image version
valiot-app	(Requires manual image build from repo).
valiot-app-db	postgres:14

Script usage info

To see a reduced version of the script documentation, run:

./wspace

To see the script full documentation, run:

./wspace help

System setup

In order to work with this repository it is necesary to install Docker Desktop on the system. The installation can be done following the documentation in this link: https://docs.docker.com/engine/install/.

Authentication

For the use of the workspace it is necesesary to have a Github account with access to the services packages, and authenticate Docker with it. To do that follow these steps:

1. Create a personal access token (classic, not fine-grained) with the scope read:packages checked. You can do this from your Github account: Creating a personal access token.

2. Once the token is generated, within the project directory run the following command, replacing <GITHUB_USER> with your Github username and <TOKEN> with the generated one:

   ./wspace login <GITHUB_USER> <TOKEN>

Workspace setup

In order to setup the workspace is necesary to download the services images, build the containers, initialize the elixir app databases, create tokens and permissions and setup those tokens into their respective service. All of this can be done with the following commands:

Initialization

Perfect for first time runs. It is the combination of the build, setup and up commands functionality:

./wspace init

It excecutes all the following configuration steps:

1. Stops and removes all workspace containers if any.
2. Downloads (if they have not been downloaded before) and builds all docker images.
3. Drops and recreates all services databases.
4. Runs migrations in all elixir app databases.
5. Creates tokens and permissions for specific services.
6. Starts all services.

It's not necesary to run this command to start all the services every time. Once the configuration is done, it will stay in that way until one or more databases are dropped or a database container is removed. The ./wspace up command is the standard way to start an already configurated workspace.

Building containers

To assemble the architecture of the workspace, there are the composing images of services that Docker downloads. However, these images must be rebuiled assigning environment variables among other minor workspace setup.

If is needed to download the images again and/or rebuild the containers, it can be done running:

./wspace build

ℹ️ If already exists a downloaded image by Docker, it will not download it again unless that image is removed.

Databases and services setup

This command drops and recreates all elixir app databases, excecutes the migration files for all of them, creates tokens and permissions for the specific services that uses the authentication tokens and generates .env files that configures those services with.

./wspace setup

Workspace deploy

All services

To start all workspace services, just run:

./wspace up

This command can receive the --console option. This option can receive one or many services and print those services console outputs. If no service is given, it will print the console output of all running services.

Single service

To start a single service (and the required ones to start it) in a specific port, run the following command.

./wspace run <SERVICE> <PORT>

Whenever the service to start is a elixir app, the command can handle passing the given <RUN_ARGS> as container initialization script arguments.

./wspace run <SERVICE> <PORT> <RUN_ARGS>

It will create an extra container for the service. The host console will become the service console output. The service process can be killed pressing Ctrl + C or Cmd ⌘ + C. The extra container will be removed after is stopped or killed.

Stop workspace

Stop service(s)

Stops the given service. If none is given, stops all workspace services.

./wspace stop <SERVICE>

Remove services

Stops all workspace services and deletes all workspace containers. This action will remove all workspace configuration setup.

./wspace down

⚠️ Warning: When this command has been run, in order to start the workspace again, the configuration setup process must be excecuted again.

Docker reset

Stops all services and purges Docker. Deletes all (not limited to Composed Workspace) downloaded images, build cache, networks, volumes and containers from Docker.

./wspace reset

⚠️ Warning: All Docker images and data will be removed. When this command has been run, in order to start the workspace again, the whole configuration setup process must be excecuted again.

Services' CLI terminal

For more advanced interaction with the servers, a CLI terminal can be accessed for any container from the Docker Desktop aplication since version 4.12: Integrated terminal for running containers.

Besides the CLI terminal itself, it also offers a console to monitor the service's console logs, the value of all environment variables and CPU and memory usage indicators.

Change UI & Worker services

Docker images

If is required to adjust or change the Docker image for ui and/or worker services, this configuration can be done in the ./workspace-compose.yml file.

To change the Docker image of a service, just change the image: parameter in their respective section.

services:
  # UI -----------------------------------------------------------------------
  ui:
    container_name: UI
    image: ghcr.io/valiot/valiot-ui:ui-demo-8961af5 # <-- Change this
  ...

  # Worker -------------------------------------------------------------------
  worker:
    container_name: Worker
    image: ghcr.io/valiot/vcos-test-worker:b14a327 # <-- Change this
  ...

After the file is updated, the ./wspace build command must be run to apply the changes to the workspace containers.

Localhost services

Its posible to use the Composed Workspace with localhost services for ui and/or worker services. The steps to make the setup in each case are explained below.

Localhost UI

1. Deploy the workspace as usual:

   ./wspace up

2. Use the URLs in the workspace menu http://localhost:4010 to set the target URLs environment varibles of the localhost ui service.

3. Start the localhost ui service.

Localhost Worker

1. Deploy the workspace as usual:

   ./wspace up

2. Stop the workspace worker service:

   ./wspace stop worker

3. Use the URLs and deployed resourses in the workspace menu http://localhost:4010 to set the environment varibles of the localhost worker service.

4. Start the localhost worker service.

Workspace for development

Valiot App

The workspace offers some developing tools for the Valiot App repository.

Start Valiot App services

To start all Valiot App services:

./wspace valiot-app.up

Build Valiot App containers

If is needed to rebuild the containers again, it can be done running:

./wspace valiot-app.build

This command can receive the --console option. This option can receive one or many services and print those services console outputs. If no service is given, it will print the console output of all running services.

Stop Valiot App services

To stop all Valiot App services, run:

./wspace valiot-app.stop

Valiot App tasks

The following commands creates an extra container for the service and when the process is done automatically exits and removes the extra container:

1. Database Setup

   To setup the valiot-app service database, navigate to the project directory and run the following command. <ENV> argument points to the environment database: dev, prod or test, if no argument is given it will set test for default.

   ./wspace valiot-db.setup <ENV=test>

2. Unit Testing

   To run the valiot-app service automated unit testing. It's possible to use all the mix test command options, like but not limited to --trace, --failed, --max_failures, --seed or --only.

   In order to validate with the automated unit tests the current Valiot App. It's needed to run the ./wspace valiot-app.setup command first.

   ./wspace valiot-app.test

The following commands creates an extra container for the service and the host console will become the service console output. The service process can be killed pressing Ctrl + C or Cmd ⌘ + C. The extra container will be removed after is stopped or killed:

1. Tests coverage

   Starts the valiot-app service and runs the mix.coveralls command to generate the unit testing coverage report. Once it have completed, starts the valiot-app service in development environment.

   ./wspace valiot-app.coverage

   In order to validate with the automated unit tests the current Valiot App. It's needed to run the ./wspace valiot-app.setup command first.

2. Start in development

   Starts the valiot-app service in development environment: MIX_ENV=dev

   ./wspace valiot-app.dev

3. Start in production

   Starts the valiot-app service in production environment: MIX_ENV=prod

   ./wspace valiot-app.prod

4. Start with Interactive Elixir

   Starts the valiot-app service in development environment within an Elixir interactive console.

   ./wspace valiot-app.iex

5. Start with custom commands

   Starts the valiot-app service passing the given <RUN_ARGS> as container initialization script arguments.

   ./wspace valiot-app.run <RUN_ARGS>