[vtkusers] vtkXMLImageDataWriter: saving large data

[Alex Malyushytskyy]

vtkDoubleArraycan be "attached" to raw C style arrays array.
You can do it if you already have an array with appropriate structure.
(You will have to figure out how to do it in C#)

check details in documentation:
void vtkDoubleArray::SetArray 	( 	double *  	array,
		vtkIdType  	size,
		int  	save	
	) 	

Alex

On Thu, Aug 18, 2011 at 1:02 AM, Oldrich Svec <oldrich.svec at gmail.com> wrote:
> Hi,
>
> thank you for your answer. You are correct that I run my simulation in
> x86 and you are right that switching to x64 would help. There is
> however a bug in F# which prevents to load x64 C# dll. So I have to
> stay with x86 for now but I have contacted Microsoft to do something
> about it :). The data comes from .NET 3D array, not from vtkImageData.
> So I hoped that I could send the data one by one from .NET directly to
> the file without any intermediate C++ array. It is just strange to me
> that .NET 3D arrays fit into memory and when I try to copy them to
> vtkDoubleArray (i.e. c++), I get an error.
>
> Lets hope, that Microsoft will answer soon and I will switch to x64 to
> avoid these problems.
>
> Thanks for your help
> Oldrich
>
>
> On Thu, Aug 18, 2011 at 3:51 AM, Alex Malyushytskyy
> <alexmalvtk at gmail.com> wrote:
>> Correct if my assumptions are wrong,
>> but since you are using .NET - you use Windows.
>>
>> Since it is unable to allocate 4194303 elements ( < INT_MAX = 2147483647 )
>> I don't think it is  limitation by index size even on 32 bit windows.
>>
>> So your application either does not:
>> -  fit in your total memory available ram+ virtual memory (in this
>> case exit other applications or increase virtual memory )
>> - or mostly likely you are running 32 bit windows and you are limited
>> by  2GB limitation (4GB total for process, half for executable code,
>> half for process data)
>>
>> If last is the case nothing you can do to increase available memory.
>>
>> About writing without duplication.
>> I am not sure about C#, but in C++ I don't think duplication is
>> required/happens if you already have vtkImageData
>>
>> vtkImageData* data = ....
>>
>> writer->SetInputConnection(data->GetProducerPort());
>> or
>> writer->SetInput( data );
>>
>> should work afaik.
>>
>> It is not clear where you get x,y,z , but if it comes from vtkImageData
>>
>> have you tried something like( I am no C# person, so syntax should be checked):
>>
>> let data  = new vtkImageData()
>> .....
>> let vti = new vtkXMLImageDataWriter()
>> vti.SetCompressorTypeToZLib()
>> vti.SetDataModeToAppended()
>> vti.SetInput() data
>> vti.SetFileName path
>> vti.Write()
>>
>>
>> Hope this helps,
>>  Alex
>>
>>
>>
>> On Thu, Aug 11, 2011 at 11:01 PM, Oldrich Svec <oldrich.svec at gmail.com> wrote:
>>> Hi,
>>>
>>> I use Activiz .NET and have 4GB of memory. I save my data during the
>>> computation in the following way:
>>>
>>> let vti = new vtkXMLImageDataWriter()
>>> vti.SetCompressorTypeToZLib()
>>> vti.SetDataModeToAppended()
>>> use arr = new vtkDoubleArray()
>>> arr.SetNumberOfComponents 1
>>> for z = 0 to data.lengthZ 0 0 - 1 do
>>>  for y = 0 to data.LengthY - 1 do
>>>    for x = 0 to data.LengthX - 1 do
>>>          arr.InsertNextValue data.[x,y,z]
>>> vti.GetInput().GetCellData().AddArray arr
>>> vti.SetFileName path
>>> vti.Write()
>>>
>>> I get an error "vtkDoubleArray: Unable to allocate 4194303
>>> elements...". Is there any way how to save the XML image data without
>>> using the intermediate vtkDoubleArray? Or is there any best practise
>>> way how to save large data into file without having to worry about the
>>> memory?
>>>
>>> Thanks in advance.
>>> Oldrich
>>> _______________________________________________
>>> Powered by www.kitware.com
>>>
>>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>>
>>> Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> http://www.vtk.org/mailman/listinfo/vtkusers
>>>
>>
>