Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add README and refactor arch #66

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
119 changes: 119 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# SealCI

SealCI is a Continuous Integration (CI) system built using Rust and designed with a microservices architecture.

## Table of contents

- [SealCI](#sealci)
- [Table of contents](#table-of-contents)
- [Dependencies](#dependencies)
- [Glossary](#glossary)
- [Architecture](#architecture)
- [Monitor](#monitor)
- [Controller](#controller)
- [Scheduler](#scheduler)
- [Agent](#agent)
- [Agent lifecycle](#agent-lifecycle)

## Dependencies

SealCI is written in Rust and makes use of the following libraries:

- actix-cors
- actix-multipart
- actix-web
- async-stream
- async-trait
- bollard
- clap
- dotenv
- env_logger
- futures
- futures-util
- lazy_static
- log
- prost
- prost-build
- reqwest
- scalar-doc
- serde
- serde_json
- serde_yaml
- sqlx
- sysinfo
- thiserror
- tokio
- tokio-stream
- tonic
- tonic-reflection
- tracing
- tracing-subscriber
- url
- yaml-rust

> You can get a similar result by running `cut -d' ' -f1 <file> |sed -r '/^\s*$/d' |sort |uniq |sed 's/^/- /'` with `<file>` containing the list of all copied dependencies from the services' `Cargo.tml`.

## Glossary

- **Action**: A CI atomic unit containing infrastructure, environment, and commands to execute.
- **Action status**: The state of the execution of an action (running, successful, failed).
- **Agent**: A computing node registered with the scheduler.
- **Agent pool**: The set of all registered agents.
- **Pipeline**: A set of actions to be executed, declared as a YAML file.
- **Scheduling**: Selection of an agent to execute an action.

For detailed documentation on each component, please refer to the respective markdown files in the `docs/arch` directory.

## Architecture

SealCI is made up of four independant microservices that serve different purpose.
They are pipelined together to create a working CI:

- The Monitor interfaces between the end user, its repository and a Controller.
- The Controller couples to a Scheduler to send actions and receive results and logs.
- The Scheduler registers Agents, sends them actions and transfers results and logs.
- The Agent executes code in the desired environment, and sends back results and logs.

Each service can be hosted, deployed and used separately.

### Monitor

The Monitor listens for specific events from remote Git repositories and triggers the controller to launch a CI process based on these events.

Features:

- Listening to events from remote Git repositories.
- Exposing a REST API to update the monitoring configuration.
- Recognizing event types and triggering pipelines accordingly.

### Controller

The Controller translates a pipeline declaration file into a list of actions to be executed. It ensures actions are executed in the correct order and provides pipeline state information.

Features:

- Users send pipelines containing actions to execute.
- Users can track actions by getting logs and states.
- The controller ensures actions are executed sequentially and handles failures.

The Controller may presently be too tightly coupled with the Scheduler.

### Scheduler

The Scheduler receives a stream of CI actions and tracks a set of CI agents. It selects agents to run the received actions based on their resource capacities and current load.

Features:

- Functional without any registered agents.
- Tracks the state and capacity of each registered agent.
- Distributes actions to agents based on resource capacities and load.

### Agent

The agent is the powerhouse of SealCI. It receives actions and runs them to complete the operational part of the CI.

#### Agent lifecycle

- **Registering with a Scheduler**: The agent registers with a scheduler and establishes a bi-directional connection. ***Described like this, it's not a loosely-coupled microservice. Which means it may not be following a good philosophy.***
- **Health and Death**: The agent streams health and status information to the scheduler.
- **Launching Actions**: The agent creates and runs a container based on the action execution environment configuration, executes commands, and cleans up after completion.
8 changes: 8 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# SealCI architecture documentation

These documents describe the services that make up SealCI's architecture and their design:

- Their definition ("What?")
- Their purpose ("Why?")
- How they are made ("How?")
- How they interact with each other
File renamed without changes.
4 changes: 3 additions & 1 deletion docs/arch/lexicon.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
## Inter-Service Lexicon
# Glossary

## Inter-service lexicon

- **Action**: a CI atomic unit. It contains infrastructure, environment and commands to execute.
- **Action status**: the state of the execution of an action (running, successful, failed).
Expand Down