[vtkusers] doubles, floats, java

[Jeff Lee]

Karin,
This is a problem in all floating point operations, namely that you are trying to represent a
base-10 number on base-2 (binary) hardware.  What you see on the screen is java's effort to
output the closest decimal representation to the double or float so
that precision is not lost during I/O.  Two common solutions are:

          a) The precision provided is better than what you need. Take what java gives you and
round
          and/or format before displaying, printing, or otherwise presenting it to humans.


          b) The precision provided is not as good as you need. If you're using Java, you can
use
          BigDecimal to get arbitrary precision at the cost of performance.

Solution a is sufficient for most, since a Java double provides something like 16 digits of
precision. I forget the
formal definition, but it's something like "The absolute value of the error will be less than
5 * 10^-17 *
actual result" .
-Jeff


Karin Faulhaber wrote:

> Hello,
>
> this is supposed to be simple, maybe I just looked at it for too long:
> I have a couple of numbers I want to put into a vtkDataArray (actually
> into a vtkPoints object). Let's say the numbers have 5 decimal places
> and 7 places (right word??) before the decimal point. For example:
> 3556236.77071
>
> But with array.GetComponent() I just don't get the same numbers I have
> put in.
>
> I played around with doubles and floats and doubleArrays and
> floatArrays, I also tried multiplicating with 100000 and casting to long
> ints and so on.
>
> Can someone please point me in the right direction? I should mention
> that I use vtk with Java (and obviously hardly have experience with this
> type and casting stuff).
>
> Thanks a lot, this is an important and basic point for me!
> Karin
>
> --
> Karin Faulhaber
> Universitaet Karlsruhe
> Institut fuer Photogrammetrie und Fernerkundung (IPF)
> Englerstr. 7
> 76128 Karlsruhe
>
> Tel: 0721/608-3676
> E-Mail: faulhaber at ipf.uni-karlsruhe.de
>
> _______________________________________________
> This is the private VTK discussion list.
> Please keep messages on-topic. Check the FAQ at: <http://public.kitware.com/cgi-bin/vtkfaq>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/vtkusers

--
J.A. Lee


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.vtk.org/pipermail/vtkusers/attachments/20010628/9926f12a/attachment.htm>