[vtkusers] vtkXMLImageDataWriter: saving large data

[Oldrich Svec]

Hi,

thank you for your answer. You are correct that I run my simulation in
x86 and you are right that switching to x64 would help. There is
however a bug in F# which prevents to load x64 C# dll. So I have to
stay with x86 for now but I have contacted Microsoft to do something
about it :). The data comes from .NET 3D array, not from vtkImageData.
So I hoped that I could send the data one by one from .NET directly to
the file without any intermediate C++ array. It is just strange to me
that .NET 3D arrays fit into memory and when I try to copy them to
vtkDoubleArray (i.e. c++), I get an error.

Lets hope, that Microsoft will answer soon and I will switch to x64 to
avoid these problems.

Thanks for your help
Oldrich


On Thu, Aug 18, 2011 at 3:51 AM, Alex Malyushytskyy
<alexmalvtk at gmail.com> wrote:
> Correct if my assumptions are wrong,
> but since you are using .NET - you use Windows.
>
> Since it is unable to allocate 4194303 elements ( < INT_MAX = 2147483647 )
> I don't think it is  limitation by index size even on 32 bit windows.
>
> So your application either does not:
> -  fit in your total memory available ram+ virtual memory (in this
> case exit other applications or increase virtual memory )
> - or mostly likely you are running 32 bit windows and you are limited
> by  2GB limitation (4GB total for process, half for executable code,
> half for process data)
>
> If last is the case nothing you can do to increase available memory.
>
> About writing without duplication.
> I am not sure about C#, but in C++ I don't think duplication is
> required/happens if you already have vtkImageData
>
> vtkImageData* data = ....
>
> writer->SetInputConnection(data->GetProducerPort());
> or
> writer->SetInput( data );
>
> should work afaik.
>
> It is not clear where you get x,y,z , but if it comes from vtkImageData
>
> have you tried something like( I am no C# person, so syntax should be checked):
>
> let data  = new vtkImageData()
> .....
> let vti = new vtkXMLImageDataWriter()
> vti.SetCompressorTypeToZLib()
> vti.SetDataModeToAppended()
> vti.SetInput() data
> vti.SetFileName path
> vti.Write()
>
>
> Hope this helps,
>  Alex
>
>
>
> On Thu, Aug 11, 2011 at 11:01 PM, Oldrich Svec <oldrich.svec at gmail.com> wrote:
>> Hi,
>>
>> I use Activiz .NET and have 4GB of memory. I save my data during the
>> computation in the following way:
>>
>> let vti = new vtkXMLImageDataWriter()
>> vti.SetCompressorTypeToZLib()
>> vti.SetDataModeToAppended()
>> use arr = new vtkDoubleArray()
>> arr.SetNumberOfComponents 1
>> for z = 0 to data.lengthZ 0 0 - 1 do
>>  for y = 0 to data.LengthY - 1 do
>>    for x = 0 to data.LengthX - 1 do
>>          arr.InsertNextValue data.[x,y,z]
>> vti.GetInput().GetCellData().AddArray arr
>> vti.SetFileName path
>> vti.Write()
>>
>> I get an error "vtkDoubleArray: Unable to allocate 4194303
>> elements...". Is there any way how to save the XML image data without
>> using the intermediate vtkDoubleArray? Or is there any best practise
>> way how to save large data into file without having to worry about the
>> memory?
>>
>> Thanks in advance.
>> Oldrich
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>
>> Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ
>>
>> Follow this link to subscribe/unsubscribe:
>> http://www.vtk.org/mailman/listinfo/vtkusers
>>
>