[vtk-developers] Charts and float/int conversion

[David Gobbi]

On Fri, Sep 24, 2010 at 9:54 AM, Marcus D. Hanwell
<marcus.hanwell at kitware.com> wrote:
> On Tue, Sep 21, 2010 at 7:11 PM, David Gobbi <david.gobbi at gmail.com> wrote:
>>
>> On Tue, Sep 21, 2010 at 4:53 PM, Jim Peterson <jimcp at cox.net> wrote:
>> > David Gobbi wrote:
>> >>
>> >> inline vtkContext2D::FloatToInt(float x)
>> >> {
>> >>  return static_cast<int>(x + 1.0625) - 1;
>> >> }
>> >>
>> >> This provides a tolerance of 1/16 of a pixel for the float-to-int
>> >> conversion. The "1" and "- 1" ensure that float values in the range
>> >> [-1.0, 0.0] are rounded down, instead of being rounded toward zero.
>> >>
>> >>  David
>> >
>> > David,
>> > pardon my ignorance, but if we are rounding float to int, why do we want
>> > -0.1 to round to -1 and not 0? in other words, why not x + 1.5 instead
>> > of x
>> > + 1.0625?
>> >
>> > Thanks,
>> > Jim
>>
>> It all depends on what the rounding is being used for.  When
>> rasterizing floating-point coordinates to the screen (i.e. to pixels),
>> it is convention to round down.
>>
>> The IEEE 754 standard describes five rounding modes, each of which is
>> valid for its intended uses:
>> 1) Round toward 0, which is C standard float-to-int
>> 2) Round towards -inf, which is like floor()
>> 3) Round towards +inf, which is like ceil()
>> 4) Round to nearest, towards even number if tie, default mode on many CPUs
>> 5) Round to nearest, away from zero if tie
>>
>> My favorite mode is round to nearest, and towards the higher of the
>> two integers in case of a tie.  Unfortunately that isn't in the
>> standard. :P
>>
> Thanks for the tips, I had been meaning to look into some of the rounding
> issues. I will test this out and seeing about getting it in.
> Marcus

I already did a bit of testing, when I was trying to find the bug that
caused the ScientificPlot test to fail on Windows.  That problem
turned out just to be a pow(10.0f, -order) where there should have
been a pow(10.0, -order) for greater precision.

I can commit my experiments as topic for you to look at.  I'll call it
charts-float-to-int.

  David