[vtkusers] Problem in Streaming

[Bernard Chiu]

Hi all,

I was trying to read in a 1GB "vti" image, process it using FFT and
write it to memory. I used streaming because the memory in my computer
is not enough for me to do this in one block. In particular, I used
vtkMemoryLimitImageDataStreamer to break up the image and hoped to do
this block by block.

However, vtkXMLImageDataReader still tries to allocate memory for the
whole image volume in each streaming iteration. Thus, there is still
not enough memory to finish the whole operation, and the following
error message comes up eventually:

ERROR: In C:\VTK504\Common\vtkDataArrayTemplate.txx, line 105
vtkDoubleArray: Unable to allocate 93603300 elements of size 8 bytes.

In particular, I found out in
vtkXMLStructuredDataReader::ReadXMLData(), the first line

this->GetOutputAsDataSet(0)->GetUpdateExtent(this->UpdateExtent);

always gets the whole extent of the image.

Please let me know whether there is any way to solve this problem.

Thanks,
Bernard


The following is my code:
(Note: ImageFFT1D is a filter derived from vtkImageFFT. It does fft in
only one dimension instead of three in vtkImageFFT)

	vtkXMLImageDataReader* vtkxmlreader = vtkXMLImageDataReader::New();
	vtkxmlreader->SetFileName((LPCSTR) inputfile);
	vtkxmlreader->ReleaseDataFlagOn();

	ImageFFT1D* ifft = ImageFFT1D::New();
	ifft->SetNumberOfThreads(1);
	ifft->SetDimensionToFFT(3);
	ifft->SetDimensionality(3);
	ifft->ForwardFFTOff();
	ifft->SetInputConnection(vtkxmlreader->GetOutputPort());
	ifft->ReleaseDataFlagOn();

	vtkMemoryLimitImageDataStreamer* streamer =
vtkMemoryLimitImageDataStreamer::New();
	streamer->SetInputConnection(ifft->GetOutputPort());
	streamer->SetMemoryLimit(50000);
	switch (XYZ)
	{
	case 0: //X -> Avoid cutting in X dim
		streamer->GetExtentTranslator()->SetSplitModeToYSlab();
		break;
	case 1: case 2://Y -> Avoid cutting in Y dim
		streamer->GetExtentTranslator()->SetSplitModeToXSlab();
		break;
	}
	streamer->UpdateWholeExtent();

	vtkXMLImageDataWriter* writer2 = vtkXMLImageDataWriter::New();
	writer2->SetInputConnection(streamer->GetOutputPort());
	writer2->SetFileName("C:\\ifft.vti");
	writer2->Write();


-- 
-------------------------------------------
Bernard Chiu, Ph.D.
Senior Fellow
Vascular Imaging Laboratory
Department of Radiology
University of Washington
815 Mercer Street
Seattle, WA, USA, 98109
Telephone: (206) 543-9249
Website: bernard-chiu.com