Jump to content
Sign in to follow this  
Domkrat

What is wrong with subtraction? Calculation mistake

Recommended Posts

It is not the operations that differ, it is the irrational numbers that suffer limitations (errors) in its binary representation. Prety much the same way you can't represent certain fractions absolutely in our familiar decimal system (ie. 1/3 = 0.33333... ad infinitum).

it looks that way because the input terms you're comparing in the operations yield or not such a representation error.

8 - 7.999 vs 0.01 + 7.99

the issue with its representation might turn up either on the input of the operation or on it's ouput, that's why it appears inconsistent (ie. conversion from decimal to binary or vice versa)

more here

Edited by gammadust

Share this post


Link to post
Share on other sites

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now
Sign in to follow this  

×