- Notifications
You must be signed in to change notification settings - Fork28
Commit43a899f
committed
Fix corner-case loss of precision in numeric ln().
When deciding on the local rscale to use for the Taylor seriesexpansion, ln_var() neglected to account for the fact that the resultis subsequently multiplied by a factor of 2^(nsqrt+1), where nsqrt isthe number of square root operations performed in the range reductionstep, which can be as high as 22 for very large inputs. This couldresult in a loss of precision, particularly when combined with largerscale values, for which a large number of Taylor series terms isrequired (up to around 400).Fix by computing a few extra digits in the Taylor series, based on theweight of the multiplicative factor log10(2^(nsqrt+1)). It remains tobe proven whether or not the other 8 extra digits used for the Taylorseries is appropriate, but this at least deals with the obviousoversight of failing to account for the effects of the finalmultiplication.Per report from Justin AnyhowStep. Reviewed by Tom Lane.Discussion:https://postgr.es/m/16280-279f299d9c06e56f@postgresql.org1 parentdba9153 commit43a899f
File tree
3 files changed
+42
-1
lines changed- src
- backend/utils/adt
- test/regress
- expected
- sql
3 files changed
+42
-1
lines changedOriginal file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
8500 | 8500 |
| |
8501 | 8501 |
| |
8502 | 8502 |
| |
| 8503 | + | |
8503 | 8504 |
| |
8504 | 8505 |
| |
8505 | 8506 |
| |
| |||
8530 | 8531 |
| |
8531 | 8532 |
| |
8532 | 8533 |
| |
| 8534 | + | |
8533 | 8535 |
| |
8534 | 8536 |
| |
8535 | 8537 |
| |
8536 | 8538 |
| |
8537 | 8539 |
| |
8538 | 8540 |
| |
| 8541 | + | |
8539 | 8542 |
| |
8540 | 8543 |
| |
8541 | 8544 |
| |
8542 | 8545 |
| |
8543 | 8546 |
| |
8544 | 8547 |
| |
8545 | 8548 |
| |
| 8549 | + | |
8546 | 8550 |
| |
8547 | 8551 |
| |
8548 | 8552 |
| |
| |||
8555 | 8559 |
| |
8556 | 8560 |
| |
8557 | 8561 |
| |
| 8562 | + | |
| 8563 | + | |
| 8564 | + | |
| 8565 | + | |
8558 | 8566 |
| |
8559 |
| - | |
| 8567 | + | |
8560 | 8568 |
| |
8561 | 8569 |
| |
8562 | 8570 |
| |
|
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
1612 | 1612 |
| |
1613 | 1613 |
| |
1614 | 1614 |
| |
| 1615 | + | |
| 1616 | + | |
| 1617 | + | |
| 1618 | + | |
| 1619 | + | |
| 1620 | + | |
| 1621 | + | |
| 1622 | + | |
| 1623 | + | |
| 1624 | + | |
| 1625 | + | |
| 1626 | + | |
| 1627 | + | |
| 1628 | + | |
| 1629 | + | |
| 1630 | + | |
| 1631 | + | |
| 1632 | + | |
| 1633 | + | |
1615 | 1634 |
| |
1616 | 1635 |
| |
1617 | 1636 |
| |
|
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
1193 | 1193 |
| |
1194 | 1194 |
| |
1195 | 1195 |
| |
| 1196 | + | |
| 1197 | + | |
| 1198 | + | |
| 1199 | + | |
| 1200 | + | |
| 1201 | + | |
| 1202 | + | |
| 1203 | + | |
| 1204 | + | |
| 1205 | + | |
| 1206 | + | |
| 1207 | + | |
| 1208 | + | |
| 1209 | + | |
1196 | 1210 |
| |
1197 | 1211 |
| |
1198 | 1212 |
| |
|
0 commit comments
Comments
(0)