Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/dev stack dockerization #335

Merged
merged 61 commits into from
Feb 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
18762f9
Build toolchain images
pavetok Jan 27, 2024
3af02e5
Fix password
pavetok Jan 27, 2024
aedb403
Revert checkout
pavetok Jan 27, 2024
38e61d6
Separate plays
pavetok Jan 27, 2024
e0c2d19
Fix builds
pavetok Jan 27, 2024
4a2a4de
Try docker step
pavetok Jan 27, 2024
5e0295b
Try docker step 2
pavetok Jan 27, 2024
fc0ea31
Try docker step 3
pavetok Jan 27, 2024
c721c72
Fix args order
pavetok Jan 27, 2024
8fdea3d
Try check sources with image
pavetok Feb 13, 2024
2c51adf
Setup buildx
pavetok Feb 13, 2024
59bb9fc
Setup java and build binaries
pavetok Feb 13, 2024
58a3660
Check daemon json
pavetok Feb 13, 2024
759429f
Setup Docker
pavetok Feb 15, 2024
bbf3165
Fix formatting
pavetok Feb 15, 2024
4ee86f4
Just append
pavetok Feb 15, 2024
ca478b8
Try bash
pavetok Feb 15, 2024
a986110
Try to debug
pavetok Feb 15, 2024
76955a6
Use whoami
pavetok Feb 15, 2024
84d536f
Use whoami 2
pavetok Feb 15, 2024
7f91fea
Hardcode user
pavetok Feb 15, 2024
7eaff39
Try tee
pavetok Feb 15, 2024
b336453
List dirs
pavetok Feb 15, 2024
dcd17b1
Setup subuid & subgid
pavetok Feb 15, 2024
f067cf6
Setup subuid & subgid
pavetok Feb 15, 2024
56b5a76
Fix jq
pavetok Feb 15, 2024
12a43a0
Try sed
pavetok Feb 15, 2024
9debc6b
Add cat
pavetok Feb 15, 2024
bca1d0a
Debug deps
pavetok Feb 15, 2024
81e579f
Setup maven
pavetok Feb 17, 2024
a03ccc7
Fix emv
pavetok Feb 17, 2024
c6ea48f
Fix conflicts
pavetok Feb 17, 2024
028a200
Log maven settings
pavetok Feb 17, 2024
c6ba829
Fix conflicts
pavetok Feb 17, 2024
68f1cad
Fix typo
pavetok Feb 17, 2024
f11aa17
Test build
pavetok Feb 17, 2024
f0d7eb0
Docker sock
pavetok Feb 17, 2024
663b2b6
Image cache key
pavetok Feb 17, 2024
954d34b
Try to fix setup maven
pavetok Feb 17, 2024
61f0802
Mount config
pavetok Feb 17, 2024
322eb64
Fix conflicts
pavetok Feb 17, 2024
fc2f5bb
Package stacks
pavetok Feb 17, 2024
b0f4d5e
Add docker sock
pavetok Feb 17, 2024
483a3e7
Rename toolchain to gear
pavetok Feb 17, 2024
cb8ebc6
Rename codebase to sources
pavetok Feb 17, 2024
ef12b85
Add setup gear composite
pavetok Feb 17, 2024
a2e2129
Fix var
pavetok Feb 17, 2024
e816e2f
Run on latest ubuntu
pavetok Feb 17, 2024
82c571a
Try experimental devenv
pavetok Feb 17, 2024
117e945
Try experimental devenv 2
pavetok Feb 17, 2024
2276101
Try experimental devenv 3
pavetok Feb 17, 2024
7987f74
Try experimental devenv 4
pavetok Feb 17, 2024
6cc483c
Try kotlin beta
pavetok Feb 17, 2024
c101c5f
Try java 21
pavetok Feb 17, 2024
10c422f
Refuse ksp
pavetok Feb 17, 2024
790a2c4
Fix linter issues
pavetok Feb 17, 2024
69c4140
Refuse konvert
pavetok Feb 17, 2024
eb94ee1
Adapt gear CID
pavetok Feb 17, 2024
01c8d4e
Rename vars
pavetok Feb 17, 2024
0d5de4c
Add verbosity
pavetok Feb 17, 2024
317311a
Increase timeout
pavetok Feb 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 0 additions & 44 deletions .github/actions/prepare/action.yaml

This file was deleted.

32 changes: 32 additions & 0 deletions .github/actions/setup-gear/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
name: Setup gear
description: Setup gear
inputs:
devenv:
description: Env name
required: true
token:
description: Github token
required: true
runs:
using: "composite"
steps:
- shell: bash
run: |
cat /etc/docker/daemon.json | jq '. + {"userns-remap": "runner"}' | sudo tee /etc/docker/daemon.json
sudo sed -i -e "1i runner:$(getent group docker | cut -d: -f3):1" /etc/subgid
sudo sed -i -e "1i runner:$(id -u):1" /etc/subuid
sudo systemctl restart docker
- uses: docker/[email protected]
- uses: docker/[email protected]
with:
context: stack/gear/${{ inputs.devenv }}
tags: stack/gear:${{ inputs.devenv }}
load: true
cache-from: type=gha,scope=stack/gear/${{ inputs.devenv }}
cache-to: type=gha,mode=max,scope=stack/gear/${{ inputs.devenv }}
- uses: docker/[email protected]
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ inputs.token }}
18 changes: 18 additions & 0 deletions .github/actions/setup-maven/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
name: Setup Maven
description: Settings & caching
runs:
using: "composite"
steps:
- uses: actions/[email protected]
with:
distribution: 'temurin'
java-version: '21' # no matter
check-latest: false
cache: 'maven'
cache-dependency-path: |
pom.xml
libs/pom.xml
tools/pom.xml
- shell: bash
run: cat ~/.m2/settings.xml
10 changes: 7 additions & 3 deletions .github/dependabot.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,16 @@ updates:
directory: "/dbs/postgres"
schedule:
interval: "weekly"
- package-ecosystem: "docker"
directory: "/stack/gear/church"
schedule:
interval: "weekly"
- package-ecosystem: "pip"
directory: "/envs/dev/church"
directory: "/stack/gear/church"
schedule:
interval: "weekly"
- package-ecosystem: "pip"
directory: "/envs/dev/turing"
directory: "/stack/gear/turing"
schedule:
interval: "weekly"
ignore:
Expand All @@ -35,7 +39,7 @@ updates:
- version-update:semver-major
- version-update:semver-minor
- package-ecosystem: "pip"
directory: "/envs/dev/godel"
directory: "/stack/gear/godel"
schedule:
interval: "weekly"
ignore:
Expand Down
135 changes: 73 additions & 62 deletions .github/workflows/beyond-doubt.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,9 @@ on:
workflow_dispatch:

env:
BINARY_STORAGE: maven.pkg.github.com
IMAGE_STORAGE: ghcr.io
IMAGE_NAME: stack/gear
BINARY_REPO: maven.pkg.github.com
IMAGE_REPO: ghcr.io

jobs:
up-to-images:
Expand All @@ -23,36 +24,43 @@ jobs:
matrix:
env:
- dev: godel
os: ubuntu-20.04
- dev: turing
os: ubuntu-22.04
- dev: church
os: ubuntu-22.04
runs-on: ${{ matrix.env.os }}
runs-on: ubuntu-latest
steps:
- uses: actions/[email protected]
- uses: ./.github/actions/prepare
- name: Run setup-gear
uses: ./.github/actions/setup-gear
with:
devenv: ${{ matrix.env.dev }}
tools: python java docker
token: ${{ secrets.GITHUB_TOKEN }}
- run: >-
ansible-playbook codebase.yaml
- name: Run setup-maven
uses: ./.github/actions/setup-maven
- name: Run ansible-playbook sources.yaml
run: >-
docker run --rm
--volume $(pwd):$(pwd)
--volume ~/.m2:/root/.m2
--workdir $(pwd)/bakers/ansible
${{ env.IMAGE_NAME }}:${{ matrix.env.dev }}
ansible-playbook sources.yaml
-e devenv=${{ matrix.env.dev }}
working-directory: bakers/ansible
env:
GITHUB_ACTOR: ${{ github.actor }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- id: images
- name: Run ansible-playbook images.yaml
id: images
run: >-
ansible-playbook images.yaml
-e binary_storage=${{ env.BINARY_STORAGE }}
-e image_storage=${{ env.IMAGE_STORAGE }}
docker run --rm
--volume $(pwd):$(pwd)
--volume ~/.m2:/root/.m2
--volume /var/run/docker.sock:/var/run/docker.sock
--volume ~/.docker/config.json:/root/.docker/config.json:ro
--env GITHUB_ACTOR=${{ github.actor }}
--env GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }}
--workdir $(pwd)/bakers/ansible
${{ env.IMAGE_NAME }}:${{ matrix.env.dev }}
ansible-playbook images.yaml -v
-e devenv=${{ matrix.env.dev }}
working-directory: bakers/ansible
env:
GITHUB_ACTOR: ${{ github.actor }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-e binary_repo=${{ env.BINARY_REPO }}
-e image_repo=${{ env.IMAGE_REPO }}
- uses: mikepenz/[email protected]
if: ${{ !cancelled() && (steps.images.outcome == 'success' || steps.images.outcome == 'failure') }}
with:
Expand Down Expand Up @@ -80,36 +88,36 @@ jobs:
purpose: [func]
env:
- dev: godel
os: ubuntu-20.04
ops: marx
- dev: turing
os: ubuntu-22.04
ops: durkheim
- dev: turing
os: ubuntu-22.04
ops: veblen
- dev: church
os: ubuntu-22.04
ops: commons
runs-on: ${{ matrix.env.os }}
runs-on: ubuntu-latest
steps:
- uses: actions/[email protected]
- uses: ./.github/actions/prepare
- name: Run setup-gear
uses: ./.github/actions/setup-gear
with:
devenv: ${{ matrix.env.dev }}
tools: python docker
token: ${{ secrets.GITHUB_TOKEN }}
- run: >-
- name: Run ansible-playbook stacks.yaml -t build
run: >-
docker run --rm
--volume $(pwd):$(pwd)
--workdir $(pwd)/bakers/ansible
${{ env.IMAGE_NAME }}:${{ matrix.env.dev }}
ansible-playbook stacks.yaml -t build
-e image_storage=${{ env.IMAGE_STORAGE }}
-e image_repo=${{ env.IMAGE_REPO }}
-e devenv=${{ matrix.env.dev }}
-e purpose=${{ matrix.purpose }}
-e opsenv=${{ matrix.env.ops }}
working-directory: bakers/ansible
- uses: actions/[email protected]
with:
name: solution-${{ matrix.purpose }}-${{ matrix.env.ops }}
path: stacks/solution/target/image-context
path: stack/solution/target/image-context
if-no-files-found: ignore

stack-testing:
Expand All @@ -123,42 +131,43 @@ jobs:
shard: [0, 1]
env:
- dev: godel
os: ubuntu-20.04
ops: marx
- dev: turing
os: ubuntu-22.04
ops: durkheim
- dev: turing
os: ubuntu-22.04
ops: veblen
- dev: church
os: ubuntu-22.04
ops: commons
runs-on: ${{ matrix.env.os }}
runs-on: ubuntu-latest
steps:
- uses: actions/[email protected]
- uses: ./.github/actions/prepare
- name: Run setup-gear
uses: ./.github/actions/setup-gear
with:
devenv: ${{ matrix.env.dev }}
tools: python java docker
token: ${{ secrets.GITHUB_TOKEN }}
- uses: actions/[email protected]
continue-on-error: true # it's ok because stack building can be skipped
with:
name: solution-${{ matrix.purpose }}-${{ matrix.env.ops }}
path: stacks/solution/target/image-context
- run: >-
path: stack/solution/target/image-context
- name: Run setup-maven
uses: ./.github/actions/setup-maven
- name: Run ansible-playbook stacks.yaml -t test
run: >-
docker run --rm
--volume $(pwd):$(pwd)
--volume ~/.m2:/root/.m2
--volume /var/run/docker.sock:/var/run/docker.sock
--workdir $(pwd)/bakers/ansible
${{ env.IMAGE_NAME }}:${{ matrix.env.dev }}
ansible-playbook stacks.yaml -t test
-e image_storage=${{ env.IMAGE_STORAGE }}
-e reminder=${{ matrix.shard }}
-e modulus=2
-e image_repo=${{ env.IMAGE_REPO }}
-e devenv=${{ matrix.env.dev }}
-e purpose=${{ matrix.purpose }}
-e opsenv=${{ matrix.env.ops }}
working-directory: bakers/ansible
env:
GITHUB_ACTOR: ${{ github.actor }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-e reminder=${{ matrix.shard }}
-e modulus=2
- uses: actions/[email protected]
if: ${{ !cancelled() }}
with:
Expand All @@ -168,47 +177,49 @@ jobs:

stack-publishing:
name: Stack publishing (${{ matrix.purpose }}, ${{ matrix.env.ops }})
timeout-minutes: 5
timeout-minutes: 10
needs: [stack-testing]
strategy:
fail-fast: false
matrix:
purpose: [func]
env:
- dev: godel
os: ubuntu-20.04
ops: marx
- dev: turing
os: ubuntu-22.04
ops: durkheim
- dev: turing
os: ubuntu-22.04
ops: veblen
- dev: church
os: ubuntu-22.04
ops: commons
runs-on: ${{ matrix.env.os }}
runs-on: ubuntu-latest
permissions:
packages: write
steps:
- uses: actions/[email protected]
- uses: ./.github/actions/prepare
- name: Run setup-gear
uses: ./.github/actions/setup-gear
with:
devenv: ${{ matrix.env.dev }}
tools: python docker
token: ${{ secrets.GITHUB_TOKEN }}
- uses: actions/[email protected]
continue-on-error: true # it's ok because stack building can be skipped
with:
name: solution-${{ matrix.purpose }}-${{ matrix.env.ops }}
path: stacks/solution/target/image-context
- run: >-
ansible-playbook stacks.yaml -t package
-e image_storage=${{ env.IMAGE_STORAGE }}
path: stack/solution/target/image-context
- name: Run ansible-playbook stacks.yaml -t package
run: >-
docker run --rm
--volume $(pwd):$(pwd)
--volume /var/run/docker.sock:/var/run/docker.sock
--volume ~/.docker/config.json:/root/.docker/config.json:ro
--workdir $(pwd)/bakers/ansible
${{ env.IMAGE_NAME }}:${{ matrix.env.dev }}
ansible-playbook stacks.yaml -t package -v
-e image_repo=${{ env.IMAGE_REPO }}
-e devenv=${{ matrix.env.dev }}
-e purpose=${{ matrix.purpose }}
-e opsenv=${{ matrix.env.ops }}
working-directory: bakers/ansible

stack-reporting:
name: Stack reporting
Expand Down
Loading
Loading