diff --git a/README.md b/README.md new file mode 100644 index 00000000..9c76d1b2 --- /dev/null +++ b/README.md @@ -0,0 +1,119 @@ +# SealCI + +SealCI is a Continuous Integration (CI) system built using Rust and designed with a microservices architecture. + +## Table of contents + +- [SealCI](#sealci) + - [Table of contents](#table-of-contents) + - [Dependencies](#dependencies) + - [Glossary](#glossary) + - [Architecture](#architecture) + - [Monitor](#monitor) + - [Controller](#controller) + - [Scheduler](#scheduler) + - [Agent](#agent) + - [Agent lifecycle](#agent-lifecycle) + +## Dependencies + +SealCI is written in Rust and makes use of the following libraries: + +- actix-cors +- actix-multipart +- actix-web +- async-stream +- async-trait +- bollard +- clap +- dotenv +- env_logger +- futures +- futures-util +- lazy_static +- log +- prost +- prost-build +- reqwest +- scalar-doc +- serde +- serde_json +- serde_yaml +- sqlx +- sysinfo +- thiserror +- tokio +- tokio-stream +- tonic +- tonic-reflection +- tracing +- tracing-subscriber +- url +- yaml-rust + +> You can get a similar result by running `cut -d' ' -f1 |sed -r '/^\s*$/d' |sort |uniq |sed 's/^/- /'` with `` containing the list of all copied dependencies from the services' `Cargo.tml`. + +## Glossary + +- **Action**: A CI atomic unit containing infrastructure, environment, and commands to execute. +- **Action status**: The state of the execution of an action (running, successful, failed). +- **Agent**: A computing node registered with the scheduler. +- **Agent pool**: The set of all registered agents. +- **Pipeline**: A set of actions to be executed, declared as a YAML file. +- **Scheduling**: Selection of an agent to execute an action. + +For detailed documentation on each component, please refer to the respective markdown files in the `docs/arch` directory. + +## Architecture + +SealCI is made up of four independant microservices that serve different purpose. +They are pipelined together to create a working CI: + +- The Monitor interfaces between the end user, its repository and a Controller. +- The Controller couples to a Scheduler to send actions and receive results and logs. +- The Scheduler registers Agents, sends them actions and transfers results and logs. +- The Agent executes code in the desired environment, and sends back results and logs. + +Each service can be hosted, deployed and used separately. + +### Monitor + +The Monitor listens for specific events from remote Git repositories and triggers the controller to launch a CI process based on these events. + +Features: + +- Listening to events from remote Git repositories. +- Exposing a REST API to update the monitoring configuration. +- Recognizing event types and triggering pipelines accordingly. + +### Controller + +The Controller translates a pipeline declaration file into a list of actions to be executed. It ensures actions are executed in the correct order and provides pipeline state information. + +Features: + +- Users send pipelines containing actions to execute. +- Users can track actions by getting logs and states. +- The controller ensures actions are executed sequentially and handles failures. + +The Controller may presently be too tightly coupled with the Scheduler. + +### Scheduler + +The Scheduler receives a stream of CI actions and tracks a set of CI agents. It selects agents to run the received actions based on their resource capacities and current load. + +Features: + +- Functional without any registered agents. +- Tracks the state and capacity of each registered agent. +- Distributes actions to agents based on resource capacities and load. + +### Agent + +The agent is the powerhouse of SealCI. It receives actions and runs them to complete the operational part of the CI. + +#### Agent lifecycle + +- **Registering with a Scheduler**: The agent registers with a scheduler and establishes a bi-directional connection. ***Described like this, it's not a loosely-coupled microservice. Which means it may not be following a good philosophy.*** +- **Health and Death**: The agent streams health and status information to the scheduler. +- **Launching Actions**: The agent creates and runs a container based on the action execution environment configuration, executes commands, and cleans up after completion. diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 00000000..ae2b137b --- /dev/null +++ b/docs/README.md @@ -0,0 +1,8 @@ +# SealCI architecture documentation + +These documents describe the services that make up SealCI's architecture and their design: + +- Their definition ("What?") +- Their purpose ("Why?") +- How they are made ("How?") +- How they interact with each other diff --git a/docs/controller.md b/docs/arch/controller.md similarity index 100% rename from docs/controller.md rename to docs/arch/controller.md diff --git a/docs/arch/lexicon.md b/docs/arch/lexicon.md index af75d548..6fd019ff 100644 --- a/docs/arch/lexicon.md +++ b/docs/arch/lexicon.md @@ -1,4 +1,6 @@ -## Inter-Service Lexicon +# Glossary + +## Inter-service lexicon - **Action**: a CI atomic unit. It contains infrastructure, environment and commands to execute. - **Action status**: the state of the execution of an action (running, successful, failed).