Why can floating-point arithmetic give mathematically incorrect results?

Floating-point keeps only a fixed number of significant digits, so adding a tiny value to a huge one can lose it entirely — which makes operations non-associative.

* Because a tiny addend is lost when added to a huge value, (a+b)-a collapses to zero while (a-a)+b preserves it, showing floating-point addition is not associative. *

float a = 1e30;
float b = 3.14159265358979;

// These SHOULD be equal:
printf("1. %.1f + %.6f - %.1f = %.6f\n", a, b, a, a+b-a);
printf("2. %.1f - %.1f + %.6f = %.6f\n", a, a, b, a-a+b);

Output:

1. 10000...0.0 + 3.141593 - 10000...0.0 = 0.000000   <- WRONG!
2. 10000...0.0 - 10000...0.0 + 3.141593 = 3.141593  <- Correct

Why (a + b - a) != b:

• Adding small b to huge a loses b (not enough precision)
• a + b effectively equals a
• Then a - a = 0

Lesson: Order of operations matters with floating-point! Floating-point is NOT associative.

Tip: When mixing large and small floats, operate on similar magnitudes first. Sort values by magnitude or use compensated summation (Kahan algorithm) for precision.

Go deeper:

• What Every Computer Scientist Should Know About Floating-Point (Goldberg) — the canonical paper on rounding error, cancellation, and why FP diverges from exact math.
• Floating-point arithmetic (Wikipedia) — IEEE 754, non-associativity of addition, and precision loss with worked examples.