[vtkusers] Floating point numerical truncation

[Bill Lorensen]

The precision is set in:
Wrapping/Tools/vtkWrapTclInit.c:            "  Tcl_SetVar2(interp,
\"tcl_precision\", (char *) NULL, \"6\", TCL_GLOBAL_ONLY);\n");

Try changing the 6 to 12 and see if that works.


On Wed, May 8, 2013 at 3:50 PM, Patrick Lowry <plowry.tx at verizon.net> wrote:

> Dear VTK User,****
>
>  ****
>
> I am using the pre-compiled binary executable for VTK 5.10 and the Tcl/Tk
> scripting language.  I also have a number of pure Tcl/Tk scripts that do
> not have VTK method calls and I had notice some time ago that VTK.exe is
> rounding floating point numbers to six significant digits.  This has
> happened since the early VTK 5.x releases and on multiple PC platforms.  It
> did not happened with the 4.x releases.  When I run the same Tcl/Tk script
> using wish85.exe no rounding occurs.  The image below shows the run window
> of the same script using VTK.exe and wish85.exe.  Both show the raw data
> line and the puts of the second two variables.  The one using VTK.exe has
> rounding and the one using wish85.exe, no rounding. Tcl/Tk has a floating
> point resolution setting that defaults to 12 significant digits.  Clearly
> VTK.exe is defaulting to 6.****
>
>  ****
>
> Can anyone direct me to a setting that will change VTK.exe to work
> correctly.****
>
>  ****
>
> Thanks,****
>
> Patrick****
>
> ****
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the VTK FAQ at:
> http://www.vtk.org/Wiki/VTK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.vtk.org/mailman/listinfo/vtkusers
>
>


-- 
Unpaid intern in BillsBasement at noware dot com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.vtk.org/pipermail/vtkusers/attachments/20130509/582f24e9/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 78995 bytes
Desc: not available
URL: <http://www.vtk.org/pipermail/vtkusers/attachments/20130509/582f24e9/attachment.jpeg>