27 Mart 2009 Cuma

The undeterminism of rounding floating points

Hi! Once upon a time in Ankara :), we were responsible for converting a ship fatigue assesment Fortran program to C++. The output of the program was a huge list of floating points and we were supposed to support 6 digit precision after the decimal point for the output. Unfortunately, due to the undeterminism of rounding of floats, we could succeed to support up to 5 digits. This is a good explanation about that: http://blogs.msdn.com/shawnhar/archive/2009/03/25/is-floating-point-math-deterministic.aspx

Hiç yorum yok:

Yorum Gönder