Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Re-enable FOSSA scan and add Codecov #14333

Merged
merged 17 commits into from
Jan 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 113 additions & 0 deletions .github/workflows/codecov.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
name: Code Coverage
on: [push, pull_request]
concurrency:
group: format('{0}-{1}', ${{ github.ref }}, 'Code Coverage')
cancel-in-progress: true

permissions: read-all

jobs:
test:
name: Code Coverage
runs-on: ubuntu-22.04

steps:
- name: Check out code
uses: actions/checkout@v3

- name: Check for changes in files relevant to code coverage
uses: frouioui/paths-filter@main
id: changes
with:
token: ''
filters: |
changed_files:
- .github/workflows/codecov.yml
- 'go/**'
- go.mod
- go.sum
- Makefile

- name: Set up Go
if: steps.changes.outputs.changed_files == 'true'
uses: actions/setup-go@v4
with:
go-version: 1.21.5

- name: Set up python
if: steps.changes.outputs.changed_files == 'true'
uses: actions/setup-python@v4

- name: Tune the OS
if: steps.changes.outputs.changed_files == 'true'
run: |
sudo sysctl -w net.ipv4.ip_local_port_range="22768 65535"
# Increase the asynchronous non-blocking I/O. More information at https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_use_native_aio
echo "fs.aio-max-nr = 1048576" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p /etc/sysctl.conf

- name: Get dependencies
if: steps.changes.outputs.changed_files == 'true'
run: |
export DEBIAN_FRONTEND="noninteractive"
sudo apt-get update

# Uninstall any previously installed MySQL first
sudo systemctl stop apparmor
sudo DEBIAN_FRONTEND="noninteractive" apt-get remove -y --purge mysql-server mysql-client mysql-common
sudo apt-get -y autoremove
sudo apt-get -y autoclean
sudo deluser mysql
sudo rm -rf /var/lib/mysql
sudo rm -rf /etc/mysql

# Get key to latest MySQL repo
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys A8D3785C

# mysql80
wget -c https://dev.mysql.com/get/mysql-apt-config_0.8.29-1_all.deb
echo mysql-apt-config mysql-apt-config/select-server select mysql-8.0 | sudo debconf-set-selections
sudo DEBIAN_FRONTEND="noninteractive" dpkg -i mysql-apt-config*
sudo apt-get update
sudo DEBIAN_FRONTEND="noninteractive" apt-get install -y mysql-server mysql-client

sudo apt-get install -y make unzip g++ curl git wget ant openjdk-11-jdk eatmydata
sudo service mysql stop
sudo bash -c "echo '/usr/sbin/mysqld { }' > /etc/apparmor.d/usr.sbin.mysqld" # https://bugs.launchpad.net/ubuntu/+source/mariadb-10.1/+bug/1806263
sudo ln -s /etc/apparmor.d/usr.sbin.mysqld /etc/apparmor.d/disable/
sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.mysqld || echo "could not remove mysqld profile"

mkdir -p dist bin
curl -L https://github.com/coreos/etcd/releases/download/v3.3.10/etcd-v3.3.10-linux-amd64.tar.gz | tar -zxC dist
mv dist/etcd-v3.3.10-linux-amd64/{etcd,etcdctl} bin/

go mod download
go install golang.org/x/tools/cmd/goimports@latest

- name: Run make tools
if: steps.changes.outputs.changed_files == 'true'
run: |
make tools

- name: Run unit tests and generate code coverage reports
if: steps.changes.outputs.changed_files == 'true'
timeout-minutes: 45
run: |
set -exo pipefail
# We set the VTDATAROOT to the /tmp folder to reduce the file path of mysql.sock file
# which musn't be more than 107 characters long.
export VTDATAROOT="/tmp/"

export NOVTADMINBUILD=1

# Exclude endtoend tests from the coverage report.
# TODO: figure out how best to include our endtoend tests in the coverage report.
rm -rf go/test/endtoend go/*/endtoend go/vt/*/endtoend go/cmd/vttestserver

eatmydata -- make unit_test_cover

- name: Upload coverage reports to codecov.io
if: steps.changes.outputs.changed_files == 'true'
uses: codecov/codecov-action@v3
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
11 changes: 11 additions & 0 deletions .github/workflows/static_checks_etc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,17 @@ jobs:
if: steps.skip-workflow.outputs.skip-workflow == 'false'
uses: actions/checkout@v3

- name: Run FOSSA scan and upload build data
# Fails on pull requests when using the API key secret.
# In order to run it on pull requests we would need to
# generate a push only token and specify that as plain
# text here:
# https://github.com/fossa-contrib/fossa-action#push-only-api-token
if: github.ref == 'refs/heads/main'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uses: fossa-contrib/fossa-action@v2
with:
fossa-api-key: ${{secrets.fossaApiKey}}

- name: Check for changes in Go files
if: steps.skip-workflow.outputs.skip-workflow == 'false'
uses: frouioui/paths-filter@main
Expand Down
12 changes: 8 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -214,10 +214,14 @@ e2e_test: build
go test $(VT_GO_PARALLEL) ./go/.../endtoend/...

# Run the code coverage tools, compute aggregate.
# If you want to improve in a directory, run:
# go test -coverprofile=coverage.out && go tool cover -html=coverage.out
unit_test_cover: build
go test $(VT_GO_PARALLEL) -cover ./go/... | misc/parse_cover.py
unit_test_cover: build dependency_check demo
source build.env
go test $(VT_GO_PARALLEL) -count=1 -coverprofile=coverage.out ./go/...
# Handle go tool cover failures due to not handling `//line` directives, which
# the goyacc compiler adds to the generated parser in sql.go. See:
# https://github.com/golang/go/issues/41222
sed -i'' -e '/^vitess.io\/vitess\/go\/vt\/sqlparser\/yaccpar/d' coverage.out
go tool $(VT_GO_PARALLEL) cover -html=coverage.out

unit_test_race: build dependency_check
tools/unit_test_race.sh
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
[![Maven Central](https://maven-badges.herokuapp.com/maven-central/io.vitess/vitess-jdbc/badge.svg)](https://maven-badges.herokuapp.com/maven-central/io.vitess/vitess-jdbc)
[![codebeat badge](https://codebeat.co/badges/51c9a056-1103-4522-9a9c-dc623821ea87)](https://codebeat.co/projects/github-com-youtube-vitess)
[![Coverage Status](https://codecov.io/gh/vitessio/vitess/branch/main/graph/badge.svg)](https://codecov.io/gh/vitessio/vitess)
[![Go Report Card](https://goreportcard.com/badge/vitess.io/vitess)](https://goreportcard.com/report/vitess.io/vitess)
[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fvitessio%2Fvitess.svg?type=shield)](https://app.fossa.io/projects/git%2Bgithub.com%2Fvitessio%2Fvitess?ref=badge_shield)
[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/1724/badge)](https://bestpractices.coreinfrastructure.org/projects/1724)
Expand Down
3 changes: 2 additions & 1 deletion go/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# README

This directory contains all the Go code for Vitess.

Most of the packages at the top level are general-purpose and are suitable
Expand All @@ -16,4 +18,3 @@ import (
topodatapb "vitess.io/vitess/go/vt/proto/topodata"
)
```

Loading