Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maintain column ordering #937

Merged
merged 1 commit into from
Dec 31, 2023
Merged

Maintain column ordering #937

merged 1 commit into from
Dec 31, 2023

Conversation

serprex
Copy link
Contributor

@serprex serprex commented Dec 29, 2023

Go maps are not ordered maps,
ie iterating a map has non determinstic order

This is causes us to lose ordering info when using using TableSchema.Columns

Instead store columns as two arrays of column names & column types,
which is used when TableSchema.Columns is nil

This way we're backwards compatible while new mirrors will have correct ordering

Fixes #888

@iskakaushik
Copy link
Contributor

also adding a test would be beneficial here IMO

@serprex
Copy link
Contributor Author

serprex commented Dec 30, 2023

already being tested by updated tests, there was initially a failure because I hadn't updated getting schema from snowflake to maintain column order

@saisrirampur
Copy link
Contributor

@serprex I did functional testing for PG, BQ and SF. Looks good!

Go maps are not ordered maps,
ie iterating a map has non determinstic order

This is causes us to lose ordering info when using using TableSchema.Columns

Instead store columns as two arrays of column names & column types,
which is used when TableSchema.Columns is nil

This way we're backwards compatible while new mirrors will have correct ordering
@serprex serprex merged commit 0e468b0 into main Dec 31, 2023
serprex added a commit that referenced this pull request Dec 31, 2023
Go maps are not ordered maps,
ie iterating a map has non determinstic order

This is causes us to lose ordering info when using using TableSchema.Columns

Instead store columns as two arrays of column names & column types,
which is used when TableSchema.Columns is nil

This way we're backwards compatible while new mirrors will have correct ordering
@serprex serprex deleted the ordered-columns branch January 4, 2024 00:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Don't mangle column ordering
3 participants