Skip to content

Commit

Permalink
[IOSP-515] [CNSMR-3231] Documentation (#52)
Browse files Browse the repository at this point in the history
* Fix doc comment

* Moved WallEView.xcodeproj one level deeper

To avoid ambiguity when using `xed .` or `vapor xcode -y`

* Fix compilation issue: missing new file from Bot/Models

* Update README for WallEView

* "How it works" + "Implementation details" sections

* Add diagrams

* GraphViz: update pack mode for nicer layout

* Fix typo in diagram name

* Apply suggestions from code review

Co-Authored-By: Daniel Spindelbauer <[email protected]>

* Add paragraph about configuration via env vars

* Apply same rewording feedbacks as in other PR

* Apply suggestions from code review

Co-Authored-By: Ilya Puchka <[email protected]>

Co-authored-by: Daniel Spindelbauer <[email protected]>
Co-authored-by: Ilya Puchka <[email protected]>
  • Loading branch information
3 people authored Jan 30, 2020
1 parent 767931e commit 036eb7f
Show file tree
Hide file tree
Showing 11 changed files with 319 additions and 19 deletions.
Empty file removed Public/.gitkeep
Empty file.
123 changes: 113 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@

A bot that monitors and manages your pull requests by ensuring they are merged when they're ready and don't stack up in your repository 🤓

### Motivation
## Motivation

Pull Requests are conceptually asynchronous and they go through a series of iterations until they are finally ready to be merged which not always happens when we are expecting, we can be waiting for CI to test it, waiting for a review, ...

That can lead to the pull request staying in the repository for longer than it needs to be and potentially stacking up with other pull requests making the integrations more time consuming and challenging.

### The notion of ready
## The notion of ready

Pull Requests should meet a specific set of criteria before being merged.

Expand All @@ -18,22 +18,125 @@ Pull Requests should meet a specific set of criteria before being merged.

Depending on the workflow of each team some of them may be disabled to suit their needs.

### How?
## How the bot works

👷‍♀️ **WIP, come back later** 👷‍♂️
Wall-E is a bot written in Swift using Vapor.

![](https://media.giphy.com/media/26ybvJNaZZKpPONEc/giphy.gif)
It works by listening to the GitHub webhook to know when labels are being added/removed and when Status Checks are updated on PRs

### Client app
### Detection of the Merge Label

When a specific label – configurable via the `MERGE_LABEL` environment variable – is added to your Pull Request, Wall-E will:

- Add that PR to a queue dedicated to the branch this PR is targeting
- Post a comment in the PR to let you know that the PR has been queued and its position in the queue

(†) Wall-E handles one serial merge queue per target branch, to allow PRs targeting different branches to be merged in parallel. Which means that each PR targeting the same branch will be merged in order one after the other, but PRs targeting different branches can be merged independently in parallel.

### Integration of a PR

When the bot dequeues the top PR from a queue, it will start its integration, which consists of the following steps:

- Merge the target branch back into the PR to ensure it's up-to-date if not
- Wait for its status checks to pass
- you can configure if you want the bot to only merge if _all_ statuses are green or only the _required_ ones via the `REQUIRES_ALL_STATUS_CHECKS` environment variable
- Once the PR is deemed ready (status checks pass, green on GitHub with the minimal number of approvals reached), it will merge the PR into the target branch (squash)
- Then it will go to the next PR in the queue

If there is a failure at some point during the integration of the PR – e.g. one of the (required) status check fails:

- the bot will post a comment with the error on the PR
- then it will remove the merge label
- and go to the next PR in the queue

If you remove the merge label from a PR that was in the queue, that PR gets removed from the queue immediately and bot goes to the next PR.

### Top Priority PRs

The bot also supports "Top Priority Labels" (configurable via the `TOP_PRIORITY_LABELS` environment variable)

When you add one of those "Top Priority" labels to your PR, the bot will ensure that this PR will be merged before any non-TP PRs targeting the same branch, by making that PR jump at the front of all the other non-top-priority PRs in the queue.

For example, if your queue already contains PRs `A`,`B`,`C`,`D`,`E` with `A` and `B` already marked with one of the Top Priority label, then adding a Top Priority label to the PR `E` will make it jump in front of `C` and `D` but still after `A` and `B`, so the queue will become `A`,`B`,`E`,`C`,`D`.

## Configuration

The bot is mainly configured via environment variables. Here are the main ones that you are at least required or recommended to provide to be able to start using this bot:

Env Var | Description
---|---
`GITHUB_ORGANIZATION`<br/>`GITHUB_REPOSITORY` | The GitHub organisation and repo name this bot will watch
`GITHUB_TOKEN` | The OAuth token to use for calls to the GitHub API
`GITHUB_WEBHOOK_SECRET` | The webhook secret to use to validate webhook payloads
`MERGE_LABEL` | The name of the label to use to add a PR to the queue
`TOP_PRIORITY_LABELS` | The name of the labels to use to mark as PR as top-priority – separate multiple label names by a comma
`REQUIRES_ALL_STATUS_CHECKS` | Defines if the bot should require _all_ status checks to be green before allowing to merge a PR, or only the ones configured as _required_ in GitHub settings (the default)

Some other environment variables allow further configuration of the bot, like values for various timeouts; for the list of them all, see [`Sources/App/Extensions/EnvironmentProperties.swift`](https://github.com/babylonhealth/Wall-E/blob/master/Sources/App/Extensions/EnvironmentProperties.swift).

## Implementation details

The whole codebase is implemented in Swift using [Vapor](https://vapor.codes/).

💡 _You can use the `vapor xcode` command to generate an `xcodeproj` project and edit the code from there._

If you need to maintain/improve the code, here are some high-level implementation details that might help you navigate the codebase.

### MergeService

`MergeService` is a service class representing a single merge queue.

- It handles the logic of the state machine for the various states and transitions to process each Pull Request in its queue in order
- It is implemented using [ReactiveFeedback](https://github.com/babylonhealth/ReactiveFeedback)

Below are some diagrams to help you visualise the ReactiveFeedback state machine logic implemented in `MergeService`:

<details>
<summary>🔃 State Diagram</summary>

![State Diagram](assets/mergebot_states.png)

</details>

<details>
<summary>▶️ Action/Feedbacks Diagrams</summary>

![Action/Feedbacks Diagrams](assets/mergebot_feedbacks.png)

</details>

### DispatchService

`DispatchService` is responsible for managing multiple `MergeService` instances, one per target queue.

- The `DispatchService` single instance is the one receiving the events from the webhook, and will dispatch them to the right instance of `MergeService` associated with the target queue of the event's PR
- If such a `MergeService` instance doesn't exist yet for that target branch, it will instantiate one.
- Idle `MergeService` instances are cleaned up after a delay of inactivity – configurable via the `IDLE_BRANCH_QUEUE_CLEANUP_DELAY` environment variable – to free up the memory

### Other

The rest of the code is mainly API calls (in `API/`) and objects modelling the API data (`Models/`)


## Client app (Menu Icon)

This repository also comes with a Client app that allows you to quickly check the state of the Merge Bot queue from the menu bar.

To install the client app:
- build `WallEView.xcodeproj` in Xcode and copy the app from build products directory to your applications directory or download the app attached to the [latest GitHub release](https://github.com/babylonhealth/Wall-E/releases)
- run `defaults write com.babylonhealth.WallEView Host <application address>` to set the url to the app
- launch the app and enjoy

- Build `WallEView/WallEView.xcodeproj` in Xcode and copy the app from build products directory to your applications directory, or download the app attached to the [latest GitHub release](https://github.com/babylonhealth/Wall-E/releases)
- Run `defaults write com.babylonhealth.WallEView Host <application address>` to set the url to the app
- Launch the app and enjoy.

Once the app has been launched, a new icon should appear in your menubar.

When opening the menu item by clicking on its icon, you can select a branch to see its associated merge queue.

To kill the app and remove the menubar icon, right-click on the icon and select "Close".

Iconography © https://dribbble.com/shots/2772860-WALL-E-Movie-Icons

### Debugging
## Debugging

Using [the ngrok tool](https://dashboard.ngrok.com/get-started) you can run the app locally and still get all incoming events from GitHub webhooks.

Expand Down
4 changes: 2 additions & 2 deletions Sources/App/Extensions/EnvironmentProperties.swift
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,12 @@ extension Environment {
return try Environment.get("GITHUB_TOKEN")
}

/// GitHub Organisation name (as seen in the github.com/<orgname>/* urls)
/// GitHub Organisation name (`<orgname>` part of `github.com/<orgname>/<repo>` url)
static func gitHubOrganization() throws -> String {
return try Environment.get("GITHUB_ORGANIZATION")
}

/// URL of GitHub Repository to run the MergeBot on
/// name of GitHub Repository to run the MergeBot on (`<repo>` part of `github.com/<orgname>/<repo>` url)
static func gitHubRepository() throws -> String {
return try Environment.get("GITHUB_REPOSITORY")
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
objects = {

/* Begin PBXBuildFile section */
38291C5223E1D8C2005F572A /* CommitState.swift in Sources */ = {isa = PBXBuildFile; fileRef = 38291C5123E1D8C2005F572A /* CommitState.swift */; };
B5BCDAE5238D8E170010DE06 /* PullRequest.swift in Sources */ = {isa = PBXBuildFile; fileRef = B5BCDAE4238D8E170010DE06 /* PullRequest.swift */; };
B5CACCED238D8CFB000D3F14 /* AppDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = B5CACCEC238D8CFB000D3F14 /* AppDelegate.swift */; };
B5CACCEF238D8CFB000D3F14 /* ViewController.swift in Sources */ = {isa = PBXBuildFile; fileRef = B5CACCEE238D8CFB000D3F14 /* ViewController.swift */; };
Expand All @@ -16,7 +17,8 @@
/* End PBXBuildFile section */

/* Begin PBXFileReference section */
B5BCDAE4238D8E170010DE06 /* PullRequest.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; name = PullRequest.swift; path = ../Sources/Bot/Models/PullRequest.swift; sourceTree = "<group>"; };
38291C5123E1D8C2005F572A /* CommitState.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = CommitState.swift; sourceTree = "<group>"; };
B5BCDAE4238D8E170010DE06 /* PullRequest.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = PullRequest.swift; sourceTree = "<group>"; };
B5CACCE9238D8CFB000D3F14 /* WallEView.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = WallEView.app; sourceTree = BUILT_PRODUCTS_DIR; };
B5CACCEC238D8CFB000D3F14 /* AppDelegate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AppDelegate.swift; sourceTree = "<group>"; };
B5CACCEE238D8CFB000D3F14 /* ViewController.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ViewController.swift; sourceTree = "<group>"; };
Expand All @@ -38,6 +40,16 @@
/* End PBXFrameworksBuildPhase section */

/* Begin PBXGroup section */
38291C5323E1D8D0005F572A /* Bot.Models */ = {
isa = PBXGroup;
children = (
B5BCDAE4238D8E170010DE06 /* PullRequest.swift */,
38291C5123E1D8C2005F572A /* CommitState.swift */,
);
name = Bot.Models;
path = ../Sources/Bot/Models;
sourceTree = "<group>";
};
B5CACCE0238D8CFB000D3F14 = {
isa = PBXGroup;
children = (
Expand All @@ -60,13 +72,13 @@
B5CACCEC238D8CFB000D3F14 /* AppDelegate.swift */,
B5CACCEE238D8CFB000D3F14 /* ViewController.swift */,
B5CACCFC238D8DA8000D3F14 /* EventMonitor.swift */,
B5BCDAE4238D8E170010DE06 /* PullRequest.swift */,
38291C5323E1D8D0005F572A /* Bot.Models */,
B5CACCF0238D8CFB000D3F14 /* Assets.xcassets */,
B5CACCF2238D8CFB000D3F14 /* Main.storyboard */,
B5CACCF5238D8CFB000D3F14 /* Info.plist */,
B5CACCF6238D8CFB000D3F14 /* WallEView.entitlements */,
);
path = WallEView;
name = WallEView;
sourceTree = "<group>";
};
/* End PBXGroup section */
Expand Down Expand Up @@ -142,6 +154,7 @@
B5CACCFD238D8DA8000D3F14 /* EventMonitor.swift in Sources */,
B5CACCEF238D8CFB000D3F14 /* ViewController.swift in Sources */,
B5BCDAE5238D8E170010DE06 /* PullRequest.swift in Sources */,
38291C5223E1D8C2005F572A /* CommitState.swift in Sources */,
B5CACCED238D8CFB000D3F14 /* AppDelegate.swift in Sources */,
);
runOnlyForDeploymentPostprocessing = 0;
Expand Down Expand Up @@ -277,12 +290,12 @@
isa = XCBuildConfiguration;
buildSettings = {
ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon;
CODE_SIGN_ENTITLEMENTS = WallEView/WallEView.entitlements;
CODE_SIGN_ENTITLEMENTS = WallEView.entitlements;
CODE_SIGN_STYLE = Manual;
COMBINE_HIDPI_IMAGES = YES;
DEVELOPMENT_TEAM = "";
ENABLE_HARDENED_RUNTIME = YES;
INFOPLIST_FILE = WallEView/Info.plist;
INFOPLIST_FILE = Info.plist;
LD_RUNPATH_SEARCH_PATHS = (
"$(inherited)",
"@executable_path/../Frameworks",
Expand All @@ -298,12 +311,12 @@
isa = XCBuildConfiguration;
buildSettings = {
ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon;
CODE_SIGN_ENTITLEMENTS = WallEView/WallEView.entitlements;
CODE_SIGN_ENTITLEMENTS = WallEView.entitlements;
CODE_SIGN_STYLE = Manual;
COMBINE_HIDPI_IMAGES = YES;
DEVELOPMENT_TEAM = "";
ENABLE_HARDENED_RUNTIME = YES;
INFOPLIST_FILE = WallEView/Info.plist;
INFOPLIST_FILE = Info.plist;
LD_RUNPATH_SEARCH_PATHS = (
"$(inherited)",
"@executable_path/../Frameworks",
Expand Down
114 changes: 114 additions & 0 deletions assets/mergebot_feedbacks.dot
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# https://edotor.net/

digraph MergeBot {
pack=true

subgraph states {
node [shape=oval][style=filled,fillcolor=lightgray]
starting [penwidth=3]
# idle
ready
integrating
runningStatusChecks
integrationFailed
"*"
}

subgraph events {
node [shape=box][style=dotted]
".pullRequestsLoaded"
".pullRequestDidChange(.include)"
".pullRequestDidChange(.exclude)"
".noMorePullRequests"
".integrationDidChangeStatus(.done)"
".integrationDidChangeStatus(.failed)"
".integrationDidChangeStatus(.updating)"
".integrate"
".statusChecksDidComplete(.passed)"
".statusChecksDidComplete(.failed)"
".statusChecksDidComplete(.timedOut)"
".integrationFailureHandled"
".retryIntegration"
}

subgraph actions {
node [shape=rect]
dequeuePR
fetchPullRequest1 [label=fetchPullRequest]
fetchPullRequest2 [label=fetchPullRequest]
fetchPullRequest3 [label=fetchPullRequest]
mergePullRequest
updatePR # merge target branch back in PR
fetchCommitStatus
fetchAllStatusChecks
includePR
excludePR
postComment
removeLabel
}

subgraph feedbacks {
color=blue

subgraph cluster_whenStarting {
label="whenStarting"
starting -> ".pullRequestsLoaded"
}

subgraph cluster_whenReady {
label="whenReady"
ready -> dequeuePR
dequeuePR -> ".noMorePullRequests" [label="nil"]
dequeuePR -> fetchPullRequest1 -> ".integrate"
}

subgraph cluster_whenIntegrating {
label="whenIntegrating"
integrating -> ".integrationDidChangeStatus(.done)" [label="pr.isMerged"]

# clean
integrating -> mergePullRequest [label="state=clean"]
mergePullRequest -> ".integrationDidChangeStatus(.done)" [label="ok"]
mergePullRequest -> ".integrationDidChangeStatus(.failed)" [label="error"]

# behind
integrating -> updatePR [label="state=behind"]
updatePR -> ".integrationDidChangeStatus(.updating)" [label="success|upToDate"]
updatePR -> ".integrationDidChangeStatus(.failed)" [label="conflict"]

# blocked|unstable
integrating -> fetchAllStatusChecks [label="state=blocked|unstable"]
fetchAllStatusChecks -> ".integrationDidChangeStatus(.updating)" [label="pending"]
fetchAllStatusChecks -> ".integrationDidChangeStatus(.failed)" [label="failure"]
fetchAllStatusChecks -> ".retryIntegration" [label="success"]

# dirty
integrating -> ".integrationDidChangeStatus(.failed)" [label="state=dirty (conflicts)"]

# unknown
integrating -> fetchPullRequest2 -> ".retryIntegration" [label="state=unknown"]
}

subgraph cluster_whenRunningStatusChecks {
label="whenRunningStatusChecks"
runningStatusChecks -> fetchPullRequest3 [label="on statusCheckObserver change"]
fetchPullRequest3 -> fetchCommitStatus
fetchCommitStatus -> ".statusChecksDidComplete(.failed)" [label="failure"]
fetchCommitStatus -> ".statusChecksDidComplete(.passed)" [label="success"]
runningStatusChecks -> ".statusChecksDidComplete(.timedOut)" [label="timeout"]
}

subgraph cluster_whenIntegrationFailed {
label="whenIntegrationFailed"
integrationFailed -> postComment -> removeLabel -> ".integrationFailureHandled"
}
}

subgraph cluster_reduceDefault {
color=blue
label="reduceDefault()"
"*" -> ".pullRequestDidChange(.include)" -> includePR
"*" -> ".pullRequestDidChange(.exclude)" -> excludePR
}

}
Binary file added assets/mergebot_feedbacks.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 036eb7f

Please sign in to comment.