[ITK-users] Management of big datasets

[Bradley Lowekamp]

Hello,

It looks like this should work.

What OS, compiler, architecture are you using? How much memory is on the system you are using?

We need to narrow down where the problem is occurring. Here are some suggestions to track down and to figure out where the problem is:

Your though of switching to a ImageFileWirter is reasonable. My goto format it mha, as I have written the most tests for that IO, and have the highest confidence there.

Does this pipeline work with smaller data? Perhaps just run your data through the BInShrinkFilter in stead of Median?

I have used most of these filters with large data, and expect them to work very well. But I have not used the ScalarImageKmeans so much. Try replacing it with the MultiOtsu.

Also write out some intermediate image to see where the image processing is going awry.

Hope that helps,
Brad

On Jan 26, 2015, at 3:44 AM, gmail <elena.bresciani87 at gmail.com> wrote:

> Hi everybody,
> 
> I’m having issues with a project that requires using 3D datasets up to 4-5 GB without subsampling. 
> The pipeline is something like this:
> 
> ImageFileReader -> IntensityWindowingImageFilterType -> MedianImageFilter -> ScalarImageKmeansImageFilter -> RelabelComponentImageFilter -> RescaleIntensityImageFilter -> ImageSeriesWriter
> 
> I was getting std::bad_alloc error but I solved this problem with  -> ReleaseDataFlagOn() for each filter of the pipeline. 
> 
> My code runs well till the end and all the images of the output stack are written but after only 170 images (of a total of 1990) are written correctly whilst the others have only black pixels.
> 
> Any guess on what could be the problem? Maybe I should use ImageFileWriter instead of ImageSeriesWriter?
> 
> Best regards
> 
> Elena
> _____________________________________
> Powered by www.kitware.com
> 
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
> 
> Kitware offers ITK Training Courses, for more information visit:
> http://www.kitware.com/products/protraining.php
> 
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
> 
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/insight-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/insight-users/attachments/20150126/c79cb0b7/attachment.html>