[vtkusers] Processing large raw(binary) files

[Alex Malyushytskyy]

>> But when I am loading file I am getting a runtime wornings and errors and
the object loads partially (sometimes just several slices). I couldn't solve
the problem by increasing shrink factor or memory limit. I also checked
extent, dimensionality and spacing, but result was the same.
>> Could you please help me.

You do not provide enough information (what warninhgs, what errors, system
you are on, RAN available etc )

So just a few comments:
If you are on win32 you are limited to 2GB for data per process by default.
It is not enough to read 6GB of data, not even counting memory required to
keep shrinked data.

Even if you are on 64 bit system the amount of memory you need may be to
large for the system you are running on.
You need approximately 12 GB of memory to be able to run your code and
better it should not count virtual memory,
otherwise process may take weeks to finish.

Regards,
    Alex




On Mon, Jul 1, 2013 at 5:09 AM, Glen <mrholmes at yandex.ru> wrote:

> Hello everyone.
>
> I have a quite large binary file (> 6 GB) which contains the data I want to
> visualize. I am trying to read and shrink it using vtkImageReader,
> vtkImageShrink3D and vtkMemoryLimitImageDataStreamer.
> The code is listed below.
>
> vtkImageReader reader = vtkImageReader.New();
> reader.SetDataScalarTypeToSignedChar();
> reader.SetDataExtent(0, extent[0], 0, extent[1], 0, extent[2]);
> reader.SetDataSpacing(spacing[0], spacing[1], spacing[2]);
> reader.SetFileDimensionality(3);
> reader.SetFileName(filePath);
>
> vtkImageShrink3D shrink = vtkImageShrink3D.New();
> shrink.SetShrinkFactors(shrinkFactor, shrinkFactor, shrinkFactor);
> shrink.AveragingOn();
> shrink.MedianOn();
> shrink.SetInputConnection(reader.GetOutputPort());
>
> vtkMemoryLimitImageDataStreamer limitStreamer =
> vtkMemoryLimitImageDataStreamer.New();
> limitStreamer.SetInputConnection(shrink.GetOutputPort());
> limitStreamer.SetMemoryLimit(memoryLimit);
> limitStreamer.UpdateWholeExtent();
>
> But when I am loading file I am getting a runtime wornings and errors and
> the object loads partially (sometimes just several slices). I couldn't
> solve
> the problem by increasing shrink factor or memory limit. I also checked
> extent, dimensionality and spacing, but result was the same.
>
> Could you please help me.
>
>
>
> --
> View this message in context:
> http://vtk.1045678.n5.nabble.com/Processing-large-raw-binary-files-tp5721699.html
> Sent from the VTK - Users mailing list archive at Nabble.com.
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the VTK FAQ at:
> http://www.vtk.org/Wiki/VTK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.vtk.org/mailman/listinfo/vtkusers
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.vtk.org/pipermail/vtkusers/attachments/20130701/bce3b514/attachment.htm>