floating point etiketine sahip kayıtlar gösteriliyor. Tüm kayıtları göster
floating point etiketine sahip kayıtlar gösteriliyor. Tüm kayıtları göster
27 Mart 2009 Cuma
The undeterminism of rounding floating points
Hi! Once upon a time in Ankara :), we were responsible for converting a ship fatigue assesment Fortran program to C++. The output of the program was a huge list of floating points and we were supposed to support 6 digit precision after the decimal point for the output. Unfortunately, due to the undeterminism of rounding of floats, we could succeed to support up to 5 digits. This is a good explanation about that: http://blogs.msdn.com/shawnhar/archive/2009/03/25/is-floating-point-math-deterministic.aspx
Kaydol:
Kayıtlar (Atom)