[vtkusers] vtkImageReader much faster than vtkImageReader2??

[David Gobbi]

I made a major typo in my last email.  I meant to say "try
repeating the tests with SetDataByteOrderToLittleEndian()"

On Thu, Jun 12, 2014 at 11:55 AM, David Gobbi <david.gobbi at gmail.com> wrote:
> Hi Marco,
>
> Try repeating the tests with SetDataByteOrderToBigEndian().  If the
> byte reordering is really the most CPU-intensive part of the read, as
> I suspect, then this should reduce both times but perhaps one more
> than the other.
>
> Other than that, I don't have any guesses.  In vtkImageReader.cxx, I
> notice that it reads into a buffer where it performs the byte swap,
> whereas vtkImageReader2.cxx doesn't use a buffer.  If this makes any
> difference at all, though, it should be in favor of vtkImageReader2
> (unless CPU cache misses are involved).
>
> The only way to be sure is to run the tests in a C++ profiler, on OS X
> I use "Instruments" but I've never done profiling on Windows.  On a
> related note, one thing that I did find via profiling is that fread()
> was 4x faster than doing read() on a C++ fstream.  But this was only
> meaningful when reading from disk cache.  When the reads were hitting
> the actual disk, the difference was negligible.
>
>   David
>
>
>
> On Thu, Jun 12, 2014 at 11:37 AM, Marco Sambin <m.sambin at gmail.com> wrote:
>> Hi David,
>>
>> first of all, thank you for your reply.
>> No, the difference is not related to disk caching issues. Here are two test
>> methods that I've written:
>>
>>
>> public static void testVTKImageReader()
>>   {
>>     long startTime = System.currentTimeMillis();
>>
>>     vtkImageReader testReaderVTK =  new vtkImageReader();
>>     testReaderVTK.FileLowerLeftOn();
>>     testReaderVTK.SetFileDimensionality(3);
>>     testReaderVTK.SetFileName("C:\\Test\\SeriesVolume.raw");
>>
>>     testReaderVTK.SetDataExtent(0, 511,
>>                           0, 511,
>>                           0, 520);
>>     testReaderVTK.SetDataSpacing(1.0, 1.0, 1.0);
>>     testReaderVTK.SetDataOrigin(0.0, 0.0, 0.0);
>>     testReaderVTK.SetDataScalarTypeToUnsignedShort();
>>     testReaderVTK.SetDataByteOrderToBigEndian();
>>     testReaderVTK.UpdateWholeExtent();
>>
>>     long endTime = System.currentTimeMillis();
>>
>>     System.out.println("Volume read by vtkImageReader. Total time millis = "
>> + (endTime - startTime));
>>   }
>>
>>   public static void testVTKImageReader2()
>>   {
>>     long startTime = System.currentTimeMillis();
>>
>>     vtkImageReader2 testReaderVTK =  new vtkImageReader2();
>>     testReaderVTK.FileLowerLeftOn();
>>     testReaderVTK.SetFileDimensionality(3);
>>     testReaderVTK.SetFileName("C:\\Test\\SeriesVolume.raw");
>>
>>     testReaderVTK.SetDataExtent(0, 511,
>>                           0, 511,
>>                           0, 520);
>>     testReaderVTK.SetDataSpacing(1.0, 1.0, 1.0);
>>     testReaderVTK.SetDataOrigin(0.0, 0.0, 0.0);
>>     testReaderVTK.SetDataScalarTypeToUnsignedShort();
>>     testReaderVTK.SetDataByteOrderToBigEndian();
>>     testReaderVTK.UpdateWholeExtent();
>>
>>     long endTime = System.currentTimeMillis();
>>
>>     System.out.println("Volume read by vtkImageReader2. Total time millis =
>> " + (endTime - startTime));
>>   }
>>
>> After doing a couple of "cold starts" in order to "warm up" disk caches, on
>> my test PC method "testVTKImageReader()" executes in around 2 seconds, while
>> method "testVTKImageReader2()" executes in around 10 seconds.
>> I've tried executing testVTKImageReader() before and testVTKImageReader2()
>> after or viceversa during the same execution session, but the result doesn't
>> change: testVTKImageReader() is always around 5 times faster than
>> testVTKImageReader2().
>>
>> Can you guess why?
>> Thanks and best regards,
>>
>> Marco Sambin
>>
>>
>>
>> On Thu, Jun 12, 2014 at 7:12 PM, David Gobbi <david.gobbi at gmail.com> wrote:
>>>
>>> Hi Marco,
>>>
>>> When used the way they are in your example code, vtkImageReader and
>>> vtkImageReader2 should be exactly the same speed.
>>>
>>> Can you explain your testing methodology in more detail?  I suspect that
>>> when you tested vtkImageReader2, it was reading from disk, and when
>>> testing vtkImageReader, it was reading from disk cache.  The only part
>>> of the "read" operation that actually requires any CPU time is the byte
>>> swapping, but that should be identical for the two readers.
>>>
>>>  - David
>>>
>>>
>>> On Thu, Jun 12, 2014 at 11:00 AM, Marco Sambin <m.sambin at gmail.com> wrote:
>>> > Hi all.
>>> > I am reading a volume made up of a set of "slices" stored in a single
>>> > large
>>> > raw file, containing just the raw pixels of each slice (unsigned short
>>> > values), one slice after the other. I know in advance image size,
>>> > position,
>>> > orientation and spacing information related to each slice.
>>> > I used to read my input raw volume through the vtkImageReader2 class, as
>>> > documentation of vtkImageReader states:
>>> >
>>> > "vtkImageReader provides methods needed to read a region from a file. It
>>> > supports both transforms and masks on the input data, but as a result is
>>> > more complicated and slower than its parent class vtkImageReader2."
>>> >
>>> > so I always assumed vtkImageReader2 to be faster than vtkImageReader to
>>> > read
>>> > a simple volume made up of a set of raw slices.
>>> > This until today, when I tried replacing my instance of vtkImageReader2
>>> > with
>>> > an instance of vtkImageReader. Guess what? My volume reading code now
>>> > executes around 10 times faster!
>>> >
>>> > Is this expected? Can you guess what may be the reason for such a
>>> > significant difference in performance between vtkImageReader2 and
>>> > vtkImageReader, the latter being much faster in my scenario (despite
>>> > what
>>> > the documentation states)?
>>> >
>>> > For completeness, I am using VTK 6.1 64-bit from Java 7 64-bit, under
>>> > Windows 7 Professional 64-bit. Here is a code fragment showing how I
>>> > configure my reader:
>>> >
>>> > [...]
>>> > imageReaderVTK =  new vtkImageReader(); // much slower if I use
>>> > "vtkImageReader2" here
>>> > imageReaderVTK.FileLowerLeftOn();
>>> > imageReaderVTK.SetFileDimensionality(3);
>>> > imageReaderVTK.SetFileName(curRawVolFile.getAbsolutePath());
>>> > imageReaderVTK.SetDataExtent(0, sliceCols - 1,
>>> >                       0, sliceRows - 1,
>>> >                       0, numOfUsedImages - 1);
>>> > imageReaderVTK.SetDataSpacing(xSpacing, ySpacing, zSpacing);
>>> > imageReaderVTK.SetDataOrigin(0.0, 0.0, 0.0);
>>> > imageReaderVTK.SetDataScalarTypeToUnsignedShort();
>>> > imageReaderVTK.SetDataByteOrderToBigEndian();
>>> > imageReaderVTK.UpdateWholeExtent();
>>> > [...]
>>> >
>>> > My typical input raw volume is made up of around 500 slices, each one
>>> > 512x512 pixels large.
>>> >
>>> > Thanks in advance for your feedback.
>>> > Best regards,
>>> >
>>> > Marco Sambin
>>
>>