Skip to content

Commit

Permalink
[412 and 408 HUBBLE ] - Automated linting (#257)
Browse files Browse the repository at this point in the history
* show current errors

* show linting errors

* new config

* CI ready

* CI final

* UpdateOrderbook unchanged
  • Loading branch information
laysabit authored Jun 24, 2024
1 parent 29ab062 commit 3825a6e
Show file tree
Hide file tree
Showing 18 changed files with 223 additions and 158 deletions.
18 changes: 9 additions & 9 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,23 @@ change is, and why it is being made, with enough context for anyone to understan

<details>
<summary>PR Checklist</summary>

### PR Structure

* [ ] This PR has reasonably narrow scope (if not, break it down into smaller PRs).
* [ ] This PR avoids mixing refactoring changes with feature changes (split into two PRs
otherwise).
* [ ] This PR's title starts with the jira ticket associated with the PR.
- [ ] This PR has reasonably narrow scope (if not, break it down into smaller PRs).
- [ ] This PR avoids mixing refactoring changes with feature changes (split into two PRs
otherwise).
- [ ] This PR's title starts with the jira ticket associated with the PR.

### Thoroughness

* [ ] This PR adds tests for the most critical parts of the new functionality or fixes.
* [ ] I've updated the README with the added features, breaking changes, new instructions on how to use the repository. I updated the description of the fuction with the changes that were made.
- [ ] This PR adds tests for the most critical parts of the new functionality or fixes.
- [ ] I've updated the README with the added features, breaking changes, new instructions on how to use the repository. I updated the description of the fuction with the changes that were made.

### Release planning

* [ ] I've decided if this PR requires a new major/minor/patch version accordingly to
[semver](https://semver.org/), and I've changed the name of the BRANCH to release/* , feature/* or patch/* .
- [ ] I've decided if this PR requires a new major/minor/patch version accordingly to
[semver](https://semver.org/), and I've changed the name of the BRANCH to release/_ , feature/_ or patch/\* .
</details>

### What
Expand Down
10 changes: 5 additions & 5 deletions .github/release-drafter.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
template: |
## What's Changed
$CHANGES
**Full Changelog**: https://github.com/$OWNER/$REPOSITORY/compare/$PREVIOUS_TAG...v$RESOLVED_VERSION
template: |
## What's Changed
$CHANGES
**Full Changelog**: https://github.com/$OWNER/$REPOSITORY/compare/$PREVIOUS_TAG...v$RESOLVED_VERSION
36 changes: 18 additions & 18 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@ name: "CodeQL"

on:
push:
branches: [ "master" ]
branches: ["master"]
pull_request:
branches: [ "master" ]
branches: ["master"]
schedule:
- cron: '42 15 * * 6'
- cron: "42 15 * * 6"

jobs:
analyze:
Expand All @@ -21,21 +21,21 @@ jobs:
fail-fast: false
matrix:
include:
- language: go
build-mode: autobuild
- language: go
build-mode: autobuild

steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Checkout repository
uses: actions/checkout@v4

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
with:
category: "/language:${{matrix.language}}"
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
with:
category: "/language:${{matrix.language}}"
39 changes: 19 additions & 20 deletions .github/workflows/internal.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,32 +2,31 @@ name: internal

on:
pull_request:
branches: [ master ]
branches: [master]

jobs:

build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1
- name: Build Internal
working-directory: internal
run: go build ./...
- name: Checkout
uses: actions/checkout@v2
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1
- name: Build Internal
working-directory: internal
run: go build ./...

unit-tests:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1
- name: Run Internal Unit Tests
working-directory: internal
run: go test -v -cover ./...
- name: Checkout
uses: actions/checkout@v2
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1
- name: Run Internal Unit Tests
working-directory: internal
run: go test -v -cover ./...
32 changes: 32 additions & 0 deletions .github/workflows/lint-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: Linting

on:
pull_request:
branches:
- master

jobs:
pre-commit:
runs-on: ubuntu-latest
if: >-
github.event.pull_request.merged == false &&
github.event.pull_request.state == 'open'
steps:
- name: Checkout code
uses: actions/checkout@v2
with:
fetch-depth: 0 # Fetch all history

- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1

- id: file_changes
uses: trilom/[email protected]
with:
output: " "

- uses: pre-commit/[email protected]
env:
extra_args: --color=always --files ${{ steps.file_changes.outputs.files}}
136 changes: 68 additions & 68 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -1,68 +1,68 @@
name: Release Drafter and Publisher

on:
pull_request:
types: [closed]

permissions:
contents: read

jobs:
new_release:
if: github.event.pull_request.merged == true
permissions:
# write permission is required to create a github release
contents: write
# write permission is required for autolabeler
# otherwise, read permission is required at least
pull-requests: write
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Get branch name
id: getbranch
run: echo ::set-output name=BRANCH::${GITHUB_HEAD_REF}

# ${{ github.ref }} was not giving v* as tag name, but refs/tags/v* instead, so It had to be abbreviated
- name: Get latest abbreviated tag
id: gettag
run: echo ::set-output name=TAG::$(git describe --tags $(git rev-list --tags --max-count=1)) # get the latest tag across all branches and put it in the output TAG

- name: Calculate next version
id: nextversion
run: |
BRANCH_NAME="${{ steps.getbranch.outputs.BRANCH }}"
CURRENT_VERSION="${{ steps.gettag.outputs.TAG }}"
CURRENT_VERSION="${CURRENT_VERSION#v}" # Remove the 'v' from the start of the version
IFS='.' read -ra VERSION_PARTS <<< "$CURRENT_VERSION"
if [[ $BRANCH_NAME =~ ^release/ ]]; then
VERSION_PARTS[0]=$((VERSION_PARTS[0] + 1))
VERSION_PARTS[1]=0
VERSION_PARTS[2]=0
elif [[ $BRANCH_NAME =~ ^feature/ ]]; then
VERSION_PARTS[1]=$((VERSION_PARTS[1] + 1))
VERSION_PARTS[2]=0
elif [[ $BRANCH_NAME =~ ^patch/ ]]; then
VERSION_PARTS[2]=$((VERSION_PARTS[2] + 1))
fi
NEXT_VERSION="v${VERSION_PARTS[0]}.${VERSION_PARTS[1]}.${VERSION_PARTS[2]}"
echo ::set-output name=NEXT_VERSION::"$NEXT_VERSION"
- name: Create and publish new tag
run: |
git tag ${{ steps.nextversion.outputs.NEXT_VERSION }}
git push origin ${{ steps.nextversion.outputs.NEXT_VERSION }}
- uses: release-drafter/release-drafter@v5
with:
commitish: master
name: "stellar-etl ${{ steps.nextversion.outputs.NEXT_VERSION }}"
tag: ${{ steps.nextversion.outputs.NEXT_VERSION }}
publish: true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
name: Release Drafter and Publisher

on:
pull_request:
types: [closed]

permissions:
contents: read

jobs:
new_release:
if: github.event.pull_request.merged == true
permissions:
# write permission is required to create a github release
contents: write
# write permission is required for autolabeler
# otherwise, read permission is required at least
pull-requests: write
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v2
with:
fetch-depth: 0

- name: Get branch name
id: getbranch
run: echo ::set-output name=BRANCH::${GITHUB_HEAD_REF}

# ${{ github.ref }} was not giving v* as tag name, but refs/tags/v* instead, so It had to be abbreviated
- name: Get latest abbreviated tag
id: gettag
run: echo ::set-output name=TAG::$(git describe --tags $(git rev-list --tags --max-count=1)) # get the latest tag across all branches and put it in the output TAG

- name: Calculate next version
id: nextversion
run: |
BRANCH_NAME="${{ steps.getbranch.outputs.BRANCH }}"
CURRENT_VERSION="${{ steps.gettag.outputs.TAG }}"
CURRENT_VERSION="${CURRENT_VERSION#v}" # Remove the 'v' from the start of the version
IFS='.' read -ra VERSION_PARTS <<< "$CURRENT_VERSION"
if [[ $BRANCH_NAME =~ ^release/ ]]; then
VERSION_PARTS[0]=$((VERSION_PARTS[0] + 1))
VERSION_PARTS[1]=0
VERSION_PARTS[2]=0
elif [[ $BRANCH_NAME =~ ^feature/ ]]; then
VERSION_PARTS[1]=$((VERSION_PARTS[1] + 1))
VERSION_PARTS[2]=0
elif [[ $BRANCH_NAME =~ ^patch/ ]]; then
VERSION_PARTS[2]=$((VERSION_PARTS[2] + 1))
fi
NEXT_VERSION="v${VERSION_PARTS[0]}.${VERSION_PARTS[1]}.${VERSION_PARTS[2]}"
echo ::set-output name=NEXT_VERSION::"$NEXT_VERSION"
- name: Create and publish new tag
run: |
git tag ${{ steps.nextversion.outputs.NEXT_VERSION }}
git push origin ${{ steps.nextversion.outputs.NEXT_VERSION }}
- uses: release-drafter/release-drafter@v5
with:
commitish: master
name: "stellar-etl ${{ steps.nextversion.outputs.NEXT_VERSION }}"
tag: ${{ steps.nextversion.outputs.NEXT_VERSION }}
publish: true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ debug
*.csr
*.key
stellar-etl
env

### Credentials checks
credentials.json
11 changes: 11 additions & 0 deletions .golangci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
linters:
disable:
- gosimple
- errcheck
- ineffassign
- staticcheck

enable:
- goimports # Check import statements are formatted according to the 'goimport' command. Reformat imports in autofix mode.
- importas # Enforces consistent import aliases.
- misspell # Finds commonly misspelled English words.
25 changes: 25 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
hooks:
- id: check-added-large-files # prevents giant files from being committed.
- id: check-case-conflict # checks for files that would conflict in case-insensitive filesystems.
- id: check-merge-conflict # checks for files that contain merge conflict strings.
- id: detect-private-key # detects the presence of private keys.
- id: end-of-file-fixer # ensures that a file is either empty, or ends with one newline.
- id: fix-byte-order-marker # removes utf-8 byte order marker.
- id: mixed-line-ending # replaces or checks mixed line ending.
- id: trailing-whitespace # trims trailing whitespace.

- repo: https://github.com/golangci/golangci-lint
rev: v1.59.1
hooks:
- id: golangci-lint
entry: golangci-lint run --fix

- repo: https://github.com/pre-commit/mirrors-prettier
rev: v3.1.0
hooks:
- id: prettier
files: \.(json|markdown|md|yaml|yml)$
language_version: 14.21.3
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@ The Stellar-ETL is a data pipeline that allows users to extract data from the hi

Pay attention, it is very important to know if your modification to this repository is a release (breaking changes), a feature (functionalities) or a patch(to fix bugs). With that information, create your branch name like this:

* ```release/<branch-name>```
* ```feature/<branch-name>```
* ```patch/<branch-name>```
- `release/<branch-name>`
- `feature/<branch-name>`
- `patch/<branch-name>`

If branch is already made, just rename it *before passing the pull request*.
If branch is already made, just rename it _before passing the pull request_.

## **Table of Contents**

Expand Down Expand Up @@ -120,7 +120,8 @@ These commands export information using the [Ledger Exporter](https://github.com

> _*NOTE:*_ Using captive-core requires a Stellar Core instance that is v20.0.0 or later. The commands use the Core instance to retrieve information about changes from the ledger. More information about the Stellar ledger information can be found [here](https://developers.stellar.org/network/horizon/api-reference/resources).
> <br> As the Stellar network grows, the Stellar Core instance has to catch up on an increasingly large amount of information. This catch-up process can add some overhead to the commands in this category. In order to avoid this overhead, run prefer processing larger ranges instead of many small ones, or use unbounded mode.
> <br><br> Recommended resources for running captive-core within a KubernetesPod:
> <br><br> Recommended resources for running captive-core within a KubernetesPod:
>
> ```
> {cpu: 3.5, memory: 20Gi, ephemeral-storage: 12Gi}
> ```
Expand Down
Loading

0 comments on commit 3825a6e

Please sign in to comment.