We have been developing writePath over the past year focused on providing customers with:
During development we used reference implementations from vendors and packages such as DuckDB, ClickHouse, and NumPy to validate aggregate correctness.
We found bugs within the reference implementations falling into one of 3 main categories: Rounding, Range, and Logic.
| Category | Reference implementation bug | How writePath addresses it |
|---|---|---|
| Rounding | Digits rounded during accumulation are lost | Values are accumulated exactly and rounded once at the end |
| Range | Numbers exceed type constraints resulting in overflow or wraparound | Integers stay integers and intermediate results never overflow when the true answer fits |
| Logic | The wrong rule is applied for constant data, infinities, zero or negative weights, or empty input | Every edge case has a defined rule that is documented per function and validated against reference implementations and exact math |
Since these are not security bugs, our approach is to submit each bug to the vendor through proper channels and then post our findings here.
After each submission and post, we will add another entry to the table at the bottom of this post.
writePath is an AWS Marketplace SaaS for sub-millisecond aggregations over the first or last N rows of your data with no pre-computation. It offers 65 high-precision aggregation functions tested against reference implementations.
| Vendor | Function | Bug Submission Link | Blog Post | Date |
|---|---|---|---|---|
| DuckDB | favg | duckdb/duckdb#27961 | DuckDB favg returns the wrong average when values differ greatly in magnitude | October 27, 2026 |
favg returns the wrong average when values differ greatly in magnitude. Over the four doubles 1, 1e100, 1, and −1e100, it returns 0.0. It should return 0.5.
SELECT favg(v) FROM (VALUES (1.0::DOUBLE), (1e100), (1.0), (-1e100)) t(v);
DuckDB 2.0.0 returns:
0.0
The correct answer is 0.5. 1e100 − 1e100 + 1 + 1 = 2. The mean of the four values is 2 / 4 = 0.5. The Python code determines the exact mean with fractions and rounds it once to a double:
from fractions import Fraction as F
def favg(values):
# the exact mean, rounded once to a double
return float(sum(F(v) for v in values) / len(values))
print(favg([1.0, 1e100, 1.0, -1e100])) # 0.5
A double can only hold the equivalent of around 16 decimal digits of precision. Adding 1 to 1e100 requires more precision.
DuckDB's favg uses Kahan summation in an attempt to prevent this precision loss. However, it only catches the loss when the running sum is the larger number in the summation step. If the running sum is the smaller number, it is unable to catch the precision loss.
| Value | Adj Val | Sum | Correction | Explanation |
|---|---|---|---|---|
| — | — | 0 | 0 | init |
| 1 | 1 | 1 | 0 | First number into accumulator |
| 1e100 | 1e100 | 1e100 | 0 | We lost the 1 here due to precision |
| 1 | 1 | 1e100 | −1 | We caught the −1 since 1e100 cancel out |
| −1e100 | −1e100 | 0 | 0 | The −1 correction is lost during addition due to precision |
Therefore the average is 0/4 = 0.
Shewchuk summation keeps the sum exact by storing it as a short list of doubles called partials.
Final Step: result = sum of the partials, rounded once
| Value | Partials | Explanation |
|---|---|---|
| — | [ ] | init |
| 1 | [1] | First number into accumulator |
| 1e100 | [1, 1e100] | The 1 can't add to 1e100 partial without losing precision, so it stays a partial |
| 1 | [2, 1e100] | The 1 is added to the 1 partial |
| −1e100 | [2] | 1e100 cancels and its partial is removed |
Result: 2, and the average is 2/4 = 0.5.
Reported to DuckDB as duckdb/duckdb#27961.
writePath's FAVG uses Shewchuk summation. No precision is lost due to the way partials are handled. The sum is rounded only once at the end, so the result is the most precise double. The function ensures results are always the same regardless of row order.
writePath is an AWS Marketplace SaaS for sub-millisecond aggregations over the first or last N rows of your data with no pre-computation. It offers 65 high-precision aggregation functions tested against reference implementations.
]]>