[Lazarus] Problem rounding values at post

Daniel Erles derles at gmail.com
Wed Jun 4 13:20:41 CEST 2014


zeljko: the rTitle is defined as DECIMAL(8,4), for sure.
Allan Registos: I need 4 decimal precision on that field. And I tried to
input 0.1200, got same wrong value inserted: 0.1199

Strange. I tried many values and y found this happens:

...
inserting 0.06 stored as 0.0599
0.07 --> 0.07
0.08 --> 0.08
0.09 --> 0.0899
0.10 --> 0.10
0.11 --> 0.11
0.12 --> 0.1199
0.13 --> 0.13
0.14 --> 0.14
0.15 --> 0.1499
0.16 --> 0.16
0.17 --> 0.17
0.18 --> 0.1799
...

there's a patron here. Could this be a bug ?

Lazarus: 1.0.10
FPC: 2.6.2
Firebird: 2.5
ZeoisLib: 7.0.4


2014-06-04 5:08 GMT-04:00 zeljko <zeljko at holobit.net>:

> On 06/04/2014 10:43 AM, Mattias Gaertner wrote:
>
>> On Wed, 04 Jun 2014 15:47:24 +0800
>> "Allan E. Registos" <allan.registos at smpc.steniel.com.ph> wrote:
>>
>>  On Wednesday, 04 June, 2014 06:03 AM, Philippe wrote:
>>>
>>>>
>>>> generally, if one needs accurate values he does not use real values
>>>> ... use integer (or word, longint, longword etc) ... then when needed
>>>> make the conversion or formating (for computing or printing) ...
>>>>
>>>>  My thinking is quite the opposite. I am not a mathematician though. :)
>>> What if we need a high degree of precision in our computation?  So we
>>> need this type of figure: #####.##### the more precision(accuracy) we
>>> need, the more digits
>>>
>>
>> If you need 5 digits, i.e. decimals: multiply by 100.000 when
>> storing and divide by 100.000 when loading the value.
>> Computers work on binary numbers and decimal 1.2 (= 1+2/10) has no exact
>> representation in binary.
>>
>
> That's why numeric type is used. Saving field as integer and divide it
> while loading is stupid if you have correct numeric type in db.
>
> zeljko
>
>
> --
> _______________________________________________
> Lazarus mailing list
> Lazarus at lists.lazarus.freepascal.org
> http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20140604/cc913993/attachment-0003.html>


More information about the Lazarus mailing list