Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): parallelize update workflow #102

Closed
wants to merge 15 commits into from
Closed

chore(ci): parallelize update workflow #102

Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
3888ba5
Update update.yml
mmorel-35 Sep 4, 2021
4de5836
Update update.yml
mmorel-35 Sep 4, 2021
8b26504
Update update.yml
mmorel-35 Sep 4, 2021
6c77541
Update update.yml
mmorel-35 Sep 4, 2021
8169e59
Update update.yml
mmorel-35 Sep 4, 2021
2747672
Update update.yml
mmorel-35 Sep 4, 2021
1dfa3b1
Update update.yml
mmorel-35 Sep 4, 2021
e5f3621
Update update.yml
mmorel-35 Sep 4, 2021
2cf1fc7
Update update.yml
mmorel-35 Sep 4, 2021
f2fb8d5
Update update.yml
mmorel-35 Sep 4, 2021
ca32ac7
Update update.yml
mmorel-35 Sep 4, 2021
1f91b61
Update update.yml
mmorel-35 Sep 4, 2021
7cea767
Update update.yml
mmorel-35 Sep 4, 2021
2927fe4
Update update.yml
mmorel-35 Sep 4, 2021
6d035c5
Update update.yml
mmorel-35 Sep 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
180 changes: 67 additions & 113 deletions .github/workflows/update.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,134 +5,88 @@ on:
workflow_dispatch:

jobs:
build:
name: Compile vuln-list-update
runs-on: ubuntu-latest
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@v2
- uses: actions/cache@v2
name: Run Cache
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1.17
- name: Build the executable
run: go build -o vuln-list-update .
- uses: actions/upload-artifact@v2
name: Upload the executable
with:
name: build
path: vuln-list-update
if-no-files-found: error
update:
name: Update repo vuln-list
needs: build
name: ${{matrix.name}}
runs-on: ubuntu-latest
strategy:
matrix:
include:
- { "name": "NVD", "args": "nvd" }
- { "name": "Alpine Issue Tracker", args: "alpine" }
- { "name": "Debian Security Bug Tracker", "args": "debian"}
- { "name": "Debian OVAL", "args": "debian-oval" }
- { "name": "Ubuntu CVE Tracker", "args": "ubuntu" }
- { "name": "Amazon Linux AMI Security Advisory", "args": "amazon" }
- { "name": "Oracle Linux OVAL", "args": "oracle-oval" }
- { "name": "Red Hat OVALv2", "args": "redhat-oval" }
- { "name": "Photon CVE Advisory", "args": "photon" }
- { "name": "GitHub Security Advisory", "args": "ghsa" }
- { "name": "CWE", "args": "cwe" }
- { "name": "SUSE CVRF", "args": "suse-cvrf" }
- { "name": "GitLab Advisory Database", "args": "glad" }
- { "name": "Arch Linux Security Advisory", "args": "arch-linux" }
- { "name": "AlmaLinux Security Advisory", "args": "alma" }
# Red Hat Security Data API is unstable.
# It should be split into small pieces to reduce the impact of failure.
- { "name": "Red Hat Security Data API 1996-2002", "args": "redhat -years 1996,1997,1998,1999,2000,2001,2002" }
- { "name": "Red Hat Security Data API 2003-2008", "args": "redhat -years 2003,2004,2005,2006,2007,2008" }
- { "name": "Red Hat Security Data API 2009-2011", "args": "redhat -years 2009,2010,2011" }
- { "name": "Red Hat Security Data API 2012", "args": "redhat -years 2012" }
- { "name": "Red Hat Security Data API 2013", "args": "redhat -years 2013" }
- { "name": "Red Hat Security Data API 2014", "args": "redhat -years 2014" }
- { "name": "Red Hat Security Data API 2015", "args": "redhat -years 2015" }
- { "name": "Red Hat Security Data API 2016", "args": "redhat -years 2016" }
- { "name": "Red Hat Security Data API 2017", "args": "redhat -years 2017" }
- { "name": "Red Hat Security Data API 2018", "args": "redhat -years 2018" }
- { "name": "Red Hat Security Data API 2019-2021", "args": "redhat -years 2019,2020,2021" }
env:
GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN }}
steps:
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1.17

- name: Check out code into the Go module directory
uses: actions/checkout@v2

- name: Set Owner
run: echo "VULNLIST_REPOSITORY_OWNER=$(echo ${GITHUB_REPOSITORY} | awk -F / '{print $1}' | sed -e 's/:refs//')" >> $GITHUB_ENV
shell: bash

- name: Setup github user email and name
run: |
git config --global user.email "[email protected]"
git config --global user.name "GitHub Action"

- name: Compile vuln-list-update
run: go build -o vuln-list-update .

- if: always()
name: NVD
run: ./vuln-list-update -target nvd

- if: always()
name: Alpine Issue Tracker
run: ./vuln-list-update -target alpine

- if: always()
name: Debian Security Bug Tracker
run: ./vuln-list-update -target debian

- if: always()
name: Debian OVAL
run: ./vuln-list-update -target debian-oval

- if: always()
name: Ubuntu CVE Tracker
run: ./vuln-list-update -target ubuntu

- if: always()
name: Amazon Linux AMI Security Advisory
run: ./vuln-list-update -target amazon

- if: always()
name: Oracle Linux OVAL
run: ./vuln-list-update -target oracle-oval

- if: always()
name: Red Hat OVALv2
run: ./vuln-list-update -target redhat-oval

- if: always()
name: Photon CVE Advisory
run: ./vuln-list-update -target photon

- if: always()
name: GitHub Security Advisory
run: ./vuln-list-update -target ghsa

- if: always()
name: CWE
run: ./vuln-list-update -target cwe

- if: always()
name: SUSE CVRF
run: ./vuln-list-update -target suse-cvrf

- if: always()
name: GitLab Advisory Database
run: ./vuln-list-update -target glad

- if: always()
name: Arch Linux Security Advisory
run: ./vuln-list-update -target arch-linux

- if: always()
name: AlmaLinux Security Advisory
run: ./vuln-list-update -target alma

# Red Hat Security Data API is unstable.
# It should be split into small pieces to reduce the impact of failure.
- if: always()
name: Red Hat Security Data API 1996-2002
run: ./vuln-list-update -target redhat -years 1996,1997,1998,1999,2000,2001,2002

- if: always()
name: Red Hat Security Data API 2003-2008
run: ./vuln-list-update -target redhat -years 2003,2004,2005,2006,2007,2008

- if: always()
name: Red Hat Security Data API 2009-2011
run: ./vuln-list-update -target redhat -years 2009,2010,2011

- if: always()
name: Red Hat Security Data API 2012
run: ./vuln-list-update -target redhat -years 2012

- if: always()
name: Red Hat Security Data API 2013
run: ./vuln-list-update -target redhat -years 2013

- if: always()
name: Red Hat Security Data API 2014
run: ./vuln-list-update -target redhat -years 2014

- if: always()
name: Red Hat Security Data API 2015
run: ./vuln-list-update -target redhat -years 2015

- if: always()
name: Red Hat Security Data API 2016
run: ./vuln-list-update -target redhat -years 2016

- if: always()
name: Red Hat Security Data API 2017
run: ./vuln-list-update -target redhat -years 2017

- if: always()
name: Red Hat Security Data API 2018
run: ./vuln-list-update -target redhat -years 2018

- if: always()
name: Red Hat Security Data API 2019-2021
run: ./vuln-list-update -target redhat -years 2019,2020,2021
- uses: actions/download-artifact@v2
name: Download the executable
with:
name: build
path: vuln-list-update
- name: Update ${{matrix.name}}
run: ./vuln-list-update -target ${{matrix.args}}