Floating point arithmetic is going to kill you

We all know that floating point arithmetic is imprecise but most of us pretend this isn’t a problem until it reaches out and bites us.

I was using Ruby Float to calculate VAT and, predictably, got bitten. The good news is that there is a simple solution.

The problem

The VAT rate in the UK is 17.5% (in Oct 2010) and so if we fire up the irb we can multiple by 0.175 (a Float in Ruby) to work out how much VAT to charge (in pence) on £105.00 as follows

> 10500 * 0.175
=> 1837.5

This all looks good, however we want to charge a whole number of pence and so let’s do it again rounding the result to the nearest integer

> (10500 * 0.175).round
=> 1837

But wait 1837.5 should round up to 1838 shouldn’t it?

> 10500 * 0.175
=> 1837.5
> (1837.5).round
=> 1838
> (10500 * 0.175).round
=> 1837

WTF! Lets look a little closer.

> 10500 * 0.175 == 1837.5
=> false
> sprintf('%.50f', 1837.5)
=> "1837.50000000000000000000000000000000000000000000000000"
> sprintf('%.50f', 10500 * 0.175)
=> "1837.49999999999977262632455676794052124023437500000000"

Aargh! So near and yet so far

The solution

Ruby has a neat solution for this.

Use a Rational instead of a Float to represent the percentage and get back to being precise.

> 10500 * Rational(175, 1000)
=> Rational(3675, 2)
> 10500 * Rational(175, 1000) == 1837.5
=> true
> sprintf('%.50f', 10500 * Rational(175, 1000))
=> "1837.50000000000000000000000000000000000000000000000000"
> (10500 * Rational(175, 1000)).round
=> 1838

Phew, all is well with the world again.

Now we can reliably use Ruby to calculate that 17.5% VAT on £105 is £18.38.

Thanks go to Tom for unravelling this knotted ball of string.