[vtk-developers] precision problem with vtkAxisActor2D on 32 bits linux

[Daniel.AGUILERA at CEA.FR]

Hi all,

 

I found a problem of precision in vtkAxisActor2D.cxx on Linux 32 bits
with gcc 3.4 or 4.1 with -O3 option :

 

At line 782 there is:

   If (fabs(sRange[0]/root - floor(sRange[0]/root)) < 0.01)

 

root is computed in another function with root=pow(10.0,rootPower)

 

if rootPower = -1 and sRange[0] = 1 we should have root = 0.1 and so
floor(sRange[0]/root) = 10

It's true without -O3 option but false with it (in that case the result
is 9)

 

This configuration is easy to check, just create a vtkXYPlotActor with Y
range = [1, 6] and you will have an Y axis in range [0.9, 5.9]

 

 

One way to avoid this problem of optimisation is to replace the line 782
by:

 

volatile double val = sRange[0]/root;

if (fabs(val - floor(val)) < 0.01)

 

This solution works on several linux distros I tried (32 and 64 bits,
x86, ia64, GNU or Intel compilers).

 

Best regards,

Daniel Aguilera

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20080321/ff95a67c/attachment.html>