Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

made CTEs and some columns use alias, eliminate whitespace #999

Merged
merged 1 commit into from
Jan 6, 2024

Conversation

heavycrystal
Copy link
Contributor

This is part of a series of PRs to address the size of our BigQuery merge statements. The maximum supported size for a single BQ query is 1MB and really wide tables with a lot of TOAST columns can cause our merge statement generator to generate queries that BigQuery cannot execute.

For this PR:

  1. Remove unnecessary whitespace in merge statements.
  2. Change our CTEs to use shorter aliases, _peerdb_deduped -> _d etc.
  3. Alias some of our frequently used raw table columns: _peerdb_unchanged_toast_columns -> _ut.

@iskakaushik iskakaushik merged commit 6a263b2 into main Jan 6, 2024
7 checks passed
@serprex serprex deleted the bq-merge-statement-smol-pt1 branch July 19, 2024 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants