[Insight-developers] Streaming with ImageWriter and IORegion

[Bill Lorensen]

Brad,

We cannot remove methods from a class that has been released.We should
do what we can to avoid their improper use.

Bill

On Fri, Dec 19, 2008 at 3:17 PM, Bradley Lowekamp
<blowekamp at mail.nih.gov> wrote:
> Hello,
> I have essentially completed these modifications. There is just more testing
> to write. All of the current non-IO streaming tests are passing (that should
> be). The only ImageIO class that was ready for streamed writing was MetaIO,
> so that is currently the only file type that is streaming.
> Option number 1, was chosen from bellow. It was the least amount of code,
> since it doesn't double the splitters.
> On the down side of these changes, I removed
> ImageFileWriter::Set/GetIORegion. These were low level methods which are not
> compatible with the ITK pipeline, and likely to cause segfaults and crashes
> if not used just right.
>
> What is the best way to contribute this code? IJ, attach the diff to Mantis?
> Just submit it to CVS, and hope CDash doesn't light up like a xmas tree? I
> think it's good quality, just may have missed some nuances of ITK or a
> compiler compatibility.
>
> I am excited to be able to process my 13GB RGB data set on my laptop :)
>
> Brad
> On Dec 18, 2008, at 12:23 PM, Bradley Lowekamp wrote:
>
> Hello,
> I am implementing streaming in ImageFileWriter, hoping that it can get on
> the fast track to making it into the repository. I have done most of the
> work in ImageFileWriter and now working out some difficulties with
> ImageIOBase. According to the design Luis wrote below, IOBase should have a
> virtual method to create a "region splitter". The difficulties with this
> lies in the differences between ImageIORegion and ImageRegion. Specifically
> ImageRegion is templated over dimension for use with Image, where
> ImageIORegion is not since ImageIO classes are not. The issues then comes
> from with ImageRegionSplitter<#> being templeted. So there becomes two
> choices I see:
>
> 1) Use ImageRegionSplitter in ImageIO, this is have the
> following declaration:
>     template <unsigned int VImageDimension>
>     typename ImageRegionSplitter<VImageDimension>::Pointer
>     ImageIOBase::GenerateWriteRegionSplitter(unsigned int
> numberOfStreamDivisions, const ImageRegion<VImageDimension>
> &largestPossibleRegion);
> It will also need a virtual helper function and do some dynamic casts, as
> well as the ugliness of some macro to create the templated object from a
> parameter.
> 2) Create a new ImageIORegionSplitter hierarchy, and duplicate the a bunch
> of code from ImageRegionSplitter. This will have a smoother interface:
> virtual ImageIORegionSpliter::Pointer
>         ImageIOBase::GenerateWriteRegionSplitter(unsigned int
> numberOfStreamDivisions, const ImageIORegion<VImageDimension>
> largestPossibleRegion);
>
> Any thought and suggestions are welcome!
> Thanks,
> Brad
> On Dec 6, 2008, at 6:25 PM, Luis Ibanez wrote:
>
>
> Hi Brad,
>
>
> The functionality of Streaming is not fully implemented
> on the ImageFileWriter.
>
>
> The design proposal for implementing this feature is to move
> (copy/paste) part of the code that you find in the itkStreamingImageFilter
> (in Code/BasicFilters) to the
> itkImageFileWriter class.
>
> In particular, the proposal is to use the itkImageRegionSplitter
> inside the ImageFileWriter.
>
> During the design discussions, one of the issues that came up
> is that only the specific ImageIO classes are qualified to know
> what would be the appropriate way to split the data in order to
> match the type of blocks that the output image file format can
> manage (e.g. DICOM can only stream slices).
>
>
> Therefore, the suggested design is that the ImageFileWriter
> will ask the ImageIO class to provide a specific instance of
> an itkImageRegionSplitter, and then it will use it in a for
> loop for generating sub-regions of the image, and request
> them from the preceding image filters.
>
>
> --
>
> Until this is implemented, then your best options, is to do
> just what you are doing: using the itkStreamingImageFilter
> just before the ImageFileWriter. The drawback of course is
> that you have to whole in memory the full-size output image
> before being able to write it to disk.
>
>
> Please let us know if you have further questions,
>
>
>    Thanks
>
>
>       Luis
>
>
> -------------------------
> Bradley Lowekamp wrote:
>
> Greetings all!
>
> I am trying to stream my data set. I see that MetaIO supports streaming for
> reading and writing. But I don't seem to be able to use it for writing. I am
> trying to stream my 13GB data set from a ImageFileReader through a
> ShrinkImageFilter then write is out with a ImageFileWriter. If I replace the
> writer with a StreamingImageFilter it works great, the reader streams and
> everything.
>
> To get the writer to stream (or not) I am doing this:
>
> shrinker->UpdateOutputInformation();
>
> RGBVolumeType::RegionType outputRegion =
> shrinker->GetOutput()->GetLargestPossibleRegion();
>
>  itk::ImageRegionSplitter<3>::Pointer splitter =
> itk::ImageRegionSplitter<3>::New();
>
>  numberOfSplits = splitter->GetNumberOfSplits(outputRegion, numberOfSplits);
>
>  writer->SetFileName( outputFilePN.GetPathName());
>   writer->SetInput(shrinker->GetOutput());
>
>  for(unsigned int i = 0; i < numberOfSplits; ++i) {
>
>      RGBVolumeType::RegionType streamRegion = splitter->GetSplit(i,
> numberOfSplits, outputRegion);
>
>      ioRegion = streamRegion; // sudo code
>
>      writer->SetIORegion(ioRegion);
>
>      writer->Update();
>
>    }
>
> Is this the correct approach? Does anyone have an example of streaming
> writing? I think I am going to dig though the ImageFileWriter now to see
> what are the updates going on in the pipeline execution. I still a bit fuzzy
> on these details, so  may miss something.
>
> Please let me know of any suggestions.
>
> Thanks,
>
> Brad
>
> ========================================================
>
> Bradley Lowekamp  Lockheed Martin Contractor for
>
> Office of High Performance Computing and Communications
>
> National Library of Medicine blowekamp at mail.nih.gov
> <mailto:blowekamp at mail.nih.gov>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
>
> Insight-users mailing list
>
> Insight-users at itk.org
>
> http://www.itk.org/mailman/listinfo/insight-users
>
> ========================================================
> Bradley Lowekamp
> Lockheed Martin Contractor for
> Office of High Performance Computing and Communications
> National Library of Medicine
> blowekamp at mail.nih.gov
>
> _______________________________________________
> Insight-developers mailing list
> Insight-developers at itk.org
> http://www.itk.org/mailman/listinfo/insight-developers
>
> ========================================================
>
> Bradley Lowekamp
>
> Lockheed Martin Contractor for
>
> Office of High Performance Computing and Communications
>
> National Library of Medicine
>
> blowekamp at mail.nih.gov
>
>
> _______________________________________________
> Insight-developers mailing list
> Insight-developers at itk.org
> http://www.itk.org/mailman/listinfo/insight-developers
>
>