- Notifications
You must be signed in to change notification settings - Fork15
Fix output precision limit for double values (issue #118)#119
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
pgSphere used its own setting (set_sphere_output_precision function)to limit the output precision of double values, that could not begreater than 15 significant digits (DBL_DIG). It introduced someproblems with dump/restore. PostgreSQL uses its own precision setting:extra_float_digits. The PostgreSQL setting allows to use more significantdigits.This patch changes the default pgSphere output behaviour to utilizePostgreSQL extra_float_digits. Now, extra_float_digits is used by default,until set_sphere_output_precision is called.The old behaviour is kept for compatibility purposes. Once,set_sphere_output_precision is called, pgSphere starts to use the oldbehaviour (read, please, issuepostgrespro#118 discussion).The patch introduces a new function - reset_sphere_output_precision.It is used to reset to the PostgreSQL behaviour after usingset_sphere_output_precision.
This is fantastic,@vitcpp! I'm not sure I understand why there are twonew |
vitcpp commentedMar 15, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Hi@esabol, once I've changed the default behaviour of the output by improving the precision, some tests show differences in numbers. It is why I've added two new test variant comparison files. But, it seems it may be a mistake. It would be better to just replace the old files. I will check. Thank you. BTW, I think to add some a new test for testing output precision with different settings. P.S. Test variant comparison files are used if the output may vary on different platforms:https://www.postgresql.org/docs/current/regress-variant.html P.P.S. I'm going to check it on 32 bit platform as well. |
The changes should definitely have tests. I was just confused as to how the new |
vitcpp commentedMar 15, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Why not put |
expected/output_precision.out - PG 10-11expected/output_precision_1.out - PG 12+
It makes sense, thank you! It seems, set_sphere_output_precision(8) is set at the top of almost all tests. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Looks good! Thank you!
vitcpp commentedMar 18, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
I've added one more commit to set extra_float_digits = 2 in epochprop and bounding_box_gist tests as proposed by@df7cb . It allows to remove expexted/bounding_box_gist_2.out but expected/epochprop_1.out is still required. The reason to create expected/epochprop_1.out is that the earlier versions output exactly 17 digits if extra_float_digits is 2, some of these digits may be meaningless. PG versions 12+ use a special function that outputs only meaningful digits even if extra_float-digits = 2, it is why in PG 12+ some numbers may contain less than 17 digits if the text representation if enough to define the double number. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Looks better!
pgSphere used its own setting (set_sphere_output_precision function) to limit the output precision of double values, that could not be greater than 15 significant digits (DBL_DIG). It introduced some problems with dump/restore. PostgreSQL uses its own precision setting: extra_float_digits. The PostgreSQL setting allows to use more significant digits.
This patch changes the default pgSphere output behaviour to utilize PostgreSQL extra_float_digits. Now, extra_float_digits is used by default, until set_sphere_output_precision is called.
The old behaviour is kept for compatibility purposes. Once, set_sphere_output_precision is called, pgSphere starts to use the old behaviour (read, please, issue#118 discussion).
The patch introduces a new function - reset_sphere_output_precision. It is used to reset to the PostgreSQL behaviour after using set_sphere_output_precision.