[vtkusers] Rounding bug when reading char scalars

Matjaz Jakopec matjaz.jakopec at acrobot.co.uk
Tue Mar 15 12:13:14 EST 2005


I reported what I think is a bug in vtkDataReader::ReadCoScalarData()  
where float values between 0.0-1.0 are converted to char values 0-255 with:

y = 255 * x

This results in rounding errors, for example:
5 / 255 = 0.0196078 (stored in a file)
0.0196078 * 255 = 4.999989 => 4 (converted to char without rounding)

A better way is to use rounding conversion:
y = 255 * x + 0.5

I noticed this because the scalars changed when loading a saved file  
(using CVS version of VTK). However, this problem might be present in  
other parts of the code that convert from 0.0-1.0 float to char.

-- 
---------------------------------------------------------------------
Dr. Matjaz Jakopec
     The Acrobot Company Ltd
     Unit 13.3.2, The Leathermarket
     Weston Street
     London SE1 3ER
Tel: +44 (0)20 7403 5866
E-mail:  matjaz.jakopec at acrobot.co.uk
www.acrobot.co.uk
---------------------------------------------------------------------



More information about the vtkusers mailing list