[vtkusers] precision problem??

[HIRAKI Hideaki]

Hello Mark,
A variable in old Tcl interpreters stored any data as a string. VTK's 
wrapping mechanism seems to convert double values to strings with 
'sprintf("%g", temp)' on passing a double to the Tcl interpreter. The 
number of digits in this conversion is not enough to keep the precision
you need. I hope the following patch may workaround the problem.

--- Wrapping/vtkWrapTcl.c~	Thu Apr 15 14:32:45 2004
+++ Wrapping/vtkWrapTcl.c	Thu Apr 15 14:43:04 2004
@@ -100,7 +100,7 @@
       fprintf(fp,"    sprintf(tempResult,\"");
       for (i = 0; i < currentFunction->HintSize; i++)
         {
-        fprintf(fp,"%%g ");
+        fprintf(fp,"%%17g ");
         }
       fprintf(fp,"\"");
       for (i = 0; i < currentFunction->HintSize; i++)
@@ -152,7 +152,7 @@
       break;
     case 1: case 7: 
       fprintf(fp,"      char tempResult[1024];\n");
-      fprintf(fp,"      sprintf(tempResult,\"%%g\",temp%i);\n",
+      fprintf(fp,"      sprintf(tempResult,\"%%17g\",temp%i);\n",
               MAX_ARGS); 
       fprintf(fp,"      Tcl_SetResult(interp, tempResult, TCL_VOLATILE);\n");
       break;


Recent versions of Tcl interpreters has methods to store double (or the 
other) values directly. You may tweak vtkWrapTcl.c more to use this 
feature.

Regards,

Hideaki Hiraki

 
At Tue, 13 Apr 2004 14:55:22 +0800, <Mark.Palmer at csiro.au> wrote:
> Hi,
> 
> I'm having a bit of trouble storing some floating point numbers with an array, a subset of the code is shown below,
> 
> vtkDoubleArray ttt$j
>      ttt$j Initialize
>      ttt$j SetName [lindex $VariableNames $j]
>      for {set i 0} {$i < $nobs} {incr i} {
>           		ttt$j InsertNextValue [set ${element}($i)] 
>           		}
> Now, a number such as 423747.4, then seems to end up as 423747 when i get around to extracting it from the array its stored in using something like
> 
> 
> [ttt$j GetValue $i]
> 
> 
> What am i doing wrong?
> 
> 
> Mark Palmer