Why does Excel get 213,458,046,676,875 x 29 wrong?

2 points by christudor 4 years ago | 2 comments

When I put =PRODUCT(213458046676875,29) into Excel, it gives me the answer 6,190,283,353,629,370.

The actual answer is 6,190,283,353,629,375.

What has happened?

lavezza 4 years ago |

Excel only stores 15 significant digits. Once you get to the 16th digit you'll just get a zero.

Example: 6,190,283,353,629,370 + 1 will still be 6,190,283,353,629,370. 6,190,283,353,629,370 + 10 will be 6,190,283,353,629,380

db48x 4 years ago |

Floating–point numbers happened. Excel uses IEEE–754 floating–point arithmetic because it is fast and ubiquitous. One of the disadvantages of doing math this way is that as you get farther away from zero, the precision of the numbers goes down. It’s generally fine because most people don’t do arithmetic on 6 quadrillion and change, they do arithmetic on smaller, more approachable numbers.

Look for a way to solve your problem using software that uses unlimited–precision arithmetic instead.