Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

replace reviewdog action with official golangci action #1062

Merged
merged 1 commit into from
Jan 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/flow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,9 @@ jobs:
run: |
./generate_protos.sh
- uses: actions/setup-go@v4
- uses: actions/setup-go@v5
with:
go-version: ">=1.21.0"
go-version: "1.21"
cache-dependency-path: flow/go.sum

- name: install gotestsum
Expand Down
43 changes: 19 additions & 24 deletions .github/workflows/golang-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,35 +5,30 @@ on:
branches: [main]
paths: [flow/**]

permissions:
contents: read

jobs:
golangci-lint:
permissions:
checks: write
contents: read
pull-requests: write
strategy:
matrix:
runner: [ubicloud-standard-4-ubuntu-2204-arm]
runs-on: ${{ matrix.runner }}
golangci:
name: lint
runs-on: [ubicloud-standard-4-ubuntu-2204-arm]
steps:
- name: checkout
uses: actions/checkout@v4
with:
submodules: recursive

- uses: actions/checkout@v4
- uses: bufbuild/[email protected]

- name: setup protos
run: |
./generate_protos.sh

- name: install lib-geos
run: |
sudo apt-get update
sudo apt-get install libgeos-dev
- uses: actions/setup-go@v5
with:
go-version: "1.21"
cache: false
- name: golangci-lint
uses: reviewdog/action-golangci-lint@v2
uses: golangci/golangci-lint-action@v3
with:
workdir: ./flow
reporter: github-pr-review
github_token: ${{ secrets.GITHUB_TOKEN }}
golangci_lint_flags: "--timeout 10m"
fail_on_error: true
env:
REVIEWDOG_TOKEN: ${{ secrets.REVIEWDOG_TOKEN }}
version: v1.55
working-directory: ./flow
args: --timeout=10m
7 changes: 3 additions & 4 deletions flow/cmd/mirror_status.go
Original file line number Diff line number Diff line change
Expand Up @@ -103,21 +103,21 @@ func (h *FlowRequestHandler) cloneTableSummary(
flowJobName string,
) ([]*protos.CloneTableSummary, error) {
q := `
SELECT
SELECT
qp.flow_name,
qr.config_proto,
MIN(qp.start_time) AS StartTime,
COUNT(*) AS NumPartitionsTotal,
COUNT(CASE WHEN qp.end_time IS NOT NULL THEN 1 END) AS NumPartitionsCompleted,
SUM(qp.rows_in_partition) FILTER (WHERE qp.end_time IS NOT NULL) AS NumRowsSynced,
AVG(EXTRACT(EPOCH FROM (qp.end_time - qp.start_time)) * 1000) FILTER (WHERE qp.end_time IS NOT NULL) AS AvgTimePerPartitionMs
FROM
FROM
peerdb_stats.qrep_partitions qp
JOIN
peerdb_stats.qrep_runs qr
ON
qp.flow_name = qr.flow_name
WHERE
WHERE
qp.flow_name ILIKE $1
GROUP BY
qp.flow_name, qr.config_proto;
Expand Down Expand Up @@ -188,7 +188,6 @@ func (h *FlowRequestHandler) cloneTableSummary(
}

cloneStatuses = append(cloneStatuses, &res)

}
return cloneStatuses, nil
}
Expand Down
2 changes: 1 addition & 1 deletion flow/connectors/bigquery/merge_stmt_generator.go
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ func (m *mergeStmtGenerator) generateDeDupedCTE() string {
}

// generateMergeStmt generates a merge statement.
func (m *mergeStmtGenerator) generateMergeStmt(unchangedToastColumns []string) string {
func (m *mergeStmtGenerator) generateMergeStmt(_unchangedToastColumns []string) string {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This parameter isn't used, so probably this is really bad for BQ where we'll run the same query 8 times. But that can be fixed in another PR

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was causing issues for a customer: #1066
fixed in the linked PR

// comma separated list of column names
columnCount := utils.TableSchemaColumns(m.normalizedTableSchema)
backtickColNames := make([]string, 0, columnCount)
Expand Down
3 changes: 3 additions & 0 deletions flow/connectors/snowflake/snowflake.go
Original file line number Diff line number Diff line change
Expand Up @@ -652,6 +652,9 @@ func (c *SnowflakeConnector) NormalizeRecords(req *model.NormalizeRecordsRequest
},
}
mergeStatement, err := mergeGen.generateMergeStmt()
if err != nil {
return err
}

startTime := time.Now()
c.logger.Info("[merge] merging records...", slog.String("destTable", tableName))
Expand Down
Loading