Loss of decimal digits when converting from REAL to DECIMAL #129
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
123456.25 requires 37 bits ( > 4 bytes) to get stored without any dataloss. REAL datatype can store upto 4B of floating point data; anything larger than that gets truncated while casting by underlying PG functions. REAL datatype in sql server relaxes this storage restriction and hence can cast the entire data without any further dataloss. Task: BABEL-3066 Signed-off-by: Satarupa Biswas [[email protected]]
bsatarupa
commented
Apr 16, 2023
| if (sql_dialect == SQL_DIALECT_TSQL && extra_float_digits > 0) | ||
| float_to_shortest_decimal_buf(val, buf); | ||
| else | ||
| snprintf(buf, sizeof(buf), "%.*g", FLT_DIG, val); | ||
|
|
There was a problem hiding this comment.
similar logic can be found in float4out(); however in float4_numeric() the same was missing.
There was a problem hiding this comment.
we need more test cases to validate that this right check. This change implies that when dialect is TSQL, always try to use extra digits for REAL datatype when casting from float4 to numeric. Is it really the case for all possible casting scenario from float4 to numeric?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
123456.25 requires 37 bits ( > 4 bytes) to get stored without any dataloss. REAL datatype can store upto 4B of floating point data; anything larger than that gets truncated while casting by underlying PG functions.
REAL datatype in sql server relaxes this storage restriction and hence can cast the entire data without any further dataloss.
Task: BABEL-3066
Signed-off-by: Satarupa Biswas [[email protected]]
By submitting this pull request, I confirm that my contribution is under the terms of the PostgreSQL license, and grant any person obtaining a copy of the contribution permission to relicense all or a portion of my contribution to the PostgreSQL License solely to contribute all or a portion of my contribution to the PostgreSQL open source project.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.