Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Round floats but not decimals in SqlLogicTests #13743

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

findepi
Copy link
Member

@findepi findepi commented Dec 12, 2024

Which issue does this PR close?

Closes #.

Rationale for this change

  • stop rounding decimal values in SLT. It's the very nature of decimal
    arithmetics that it should in general be exact.
  • round float values taking into account the bit width of the float. The
    Float64 carries more information than Float32 or Float16.

What changes are included in this PR?

  • SLT formatting

Are these changes tested?

yes

Are there any user-facing changes?

no

- add `decimal` to function name
- drop `precision` parameter as it is not supposed to affect the result
Utilize new `to_plain_string` function
@findepi findepi requested review from alamb and jonahgao December 12, 2024 14:37
@github-actions github-actions bot added the sqllogictest SQL Logic Tests (.slt) label Dec 12, 2024
@findepi
Copy link
Member Author

findepi commented Dec 12, 2024

cc @gliga

@findepi findepi force-pushed the findepi/preserve-decimal-slt branch from 49466f2 to e35718d Compare December 12, 2024 14:42
Before the change, the request to use PostgreSQL was simply ignored when
`--complete` flag was present.
- stop rounding decimal values in SLT. It's the very nature of decimal
  arithmetics that it should in general be exact.
- round float values taking into account the bit width of the float. The
  Float64 carries more information than Float32 or Float16.
@findepi findepi force-pushed the findepi/preserve-decimal-slt branch from e35718d to b2c20bf Compare December 12, 2024 16:34
@findepi findepi marked this pull request as draft December 12, 2024 17:45
@findepi
Copy link
Member Author

findepi commented Dec 12, 2024

Bummer. We get types different from PostgreSQL, so it's hard to get results rendered the same if rendering is type dependent.

@Omega359
Copy link
Contributor

I think this will impact me significantly with the sqlite test file integration I'm working on.

@findepi
Copy link
Member Author

findepi commented Dec 13, 2024

in a positive or negative manner?

@Omega359
Copy link
Contributor

in a positive or negative manner?

Unsure @findepi. Below is an example of a variety of discrepancies between DataFusion and Postgresql results as an example of what I'm trying to deal with (df on the left, pg on the right)

image

@findepi
Copy link
Member Author

findepi commented Dec 14, 2024

float rounding could help with some of these eg

image image

However, the reason why i didn't finish this PR is that even for existing modest number of tests this is a problem.
This is because we get types different. For example 2.0 is apparently decimal in PostgreSQL, while it's a float in DF. Maybe addressing #12817 and this PR would make landing sqlite slt tests easier?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants