[vtkusers] Rounding bug when reading char scalars
Matjaz Jakopec
matjaz.jakopec at acrobot.co.uk
Tue Mar 15 12:13:14 EST 2005
I reported what I think is a bug in vtkDataReader::ReadCoScalarData()
where float values between 0.0-1.0 are converted to char values 0-255 with:
y = 255 * x
This results in rounding errors, for example:
5 / 255 = 0.0196078 (stored in a file)
0.0196078 * 255 = 4.999989 => 4 (converted to char without rounding)
A better way is to use rounding conversion:
y = 255 * x + 0.5
I noticed this because the scalars changed when loading a saved file
(using CVS version of VTK). However, this problem might be present in
other parts of the code that convert from 0.0-1.0 float to char.
--
---------------------------------------------------------------------
Dr. Matjaz Jakopec
The Acrobot Company Ltd
Unit 13.3.2, The Leathermarket
Weston Street
London SE1 3ER
Tel: +44 (0)20 7403 5866
E-mail: matjaz.jakopec at acrobot.co.uk
www.acrobot.co.uk
---------------------------------------------------------------------
More information about the vtkusers
mailing list