[Insight-developers] Streaming with ImageWriter and IORegion
Bill Lorensen
bill.lorensen at gmail.com
Mon Dec 22 10:01:22 EST 2008
Brad,
Sounds great.
Bill
On Mon, Dec 22, 2008 at 9:08 AM, Bradley Lowekamp
<blowekamp at mail.nih.gov> wrote:
> Bill,
> I'll implement this method how I think is should work. That is I'll make is
> so that it requests the ioregion, up the pipeline and will tolerate if the
> exact region is not supplied. This will maintain the backwards compatibility
> of the interface, but the behavior may be a little different.
> Then I will submit the changes into mantis so that they can be quickly
> reviewed or what not.
>
> Brad
> On Dec 19, 2008, at 6:48 PM, Bill Lorensen wrote:
>
> Brad,
>
> Really, we don't know what customers may be using this method. Maybe
> even correctly in their own ImageIO class. Many customers write their
> own ImageIO classes. For example, Slicer3 has two of its own ImageIO
> classes. To be honest, I have not looked at how it's used in the
> ImageFileWriter or current ImageIO classes. I'll take a look over the
> next few days.
>
> But, as I said before, we released the code with these methods. Since
> we made the mistake, the burden is on us to fix it, not our customers.
>
> Bill
>
> On Fri, Dec 19, 2008 at 5:05 PM, Bradley Lowekamp
> <blowekamp at mail.nih.gov> wrote:
>
> Bill,
>
> This is a very problematic method; I have strong doubts that anyone is using
>
> it. It is unclear to me what the appropriate behavior should be. If it's
>
> current behavior must be maintained then, we should copy the current Write
>
> method, so that it can be called only when needed if this method has been
>
> set, Then depreciate the method so that it can be removed later.
>
> MetaImageIO is the only class that can support this method. Use it with any
>
> other ImageIO (I think) results in no image, or undefined behavior. I
>
> believe that this method was a start of implementing streaming that was
>
> never finished. For it to be used the input to ImageFileWrite must have
>
> already executed and generated this exact region or else the results would
>
> be wrong, and undefined. Some of these problems could be reduced if
>
> ImageFileWrite copies the regionIO to a buffer/cache if needed, then it
>
> would be up to the ImageIO to do what is asked or barf.
>
> I would argue that anyone using this method should not be using
>
> ImageFileWriter and should just use the ImageIO object directly, with it's
>
> current behavior, but that's not backwardly compatible.
>
> What's be best solution?
>
> Brad
>
> On Dec 19, 2008, at 3:59 PM, Bill Lorensen wrote:
>
> Brad,
>
> We cannot remove methods from a class that has been released.We should
>
> do what we can to avoid their improper use.
>
> Bill
>
> On Fri, Dec 19, 2008 at 3:17 PM, Bradley Lowekamp
>
> <blowekamp at mail.nih.gov> wrote:
>
> Hello,
>
> I have essentially completed these modifications. There is just more testing
>
> to write. All of the current non-IO streaming tests are passing (that should
>
> be). The only ImageIO class that was ready for streamed writing was MetaIO,
>
> so that is currently the only file type that is streaming.
>
> Option number 1, was chosen from bellow. It was the least amount of code,
>
> since it doesn't double the splitters.
>
> On the down side of these changes, I removed
>
> ImageFileWriter::Set/GetIORegion. These were low level methods which are not
>
> compatible with the ITK pipeline, and likely to cause segfaults and crashes
>
> if not used just right.
>
> What is the best way to contribute this code? IJ, attach the diff to Mantis?
>
> Just submit it to CVS, and hope CDash doesn't light up like a xmas tree? I
>
> think it's good quality, just may have missed some nuances of ITK or a
>
> compiler compatibility.
>
> I am excited to be able to process my 13GB RGB data set on my laptop :)
>
> Brad
>
> On Dec 18, 2008, at 12:23 PM, Bradley Lowekamp wrote:
>
> Hello,
>
> I am implementing streaming in ImageFileWriter, hoping that it can get on
>
> the fast track to making it into the repository. I have done most of the
>
> work in ImageFileWriter and now working out some difficulties with
>
> ImageIOBase. According to the design Luis wrote below, IOBase should have a
>
> virtual method to create a "region splitter". The difficulties with this
>
> lies in the differences between ImageIORegion and ImageRegion. Specifically
>
> ImageRegion is templated over dimension for use with Image, where
>
> ImageIORegion is not since ImageIO classes are not. The issues then comes
>
> from with ImageRegionSplitter<#> being templeted. So there becomes two
>
> choices I see:
>
> 1) Use ImageRegionSplitter in ImageIO, this is have the
>
> following declaration:
>
> template <unsigned int VImageDimension>
>
> typename ImageRegionSplitter<VImageDimension>::Pointer
>
> ImageIOBase::GenerateWriteRegionSplitter(unsigned int
>
> numberOfStreamDivisions, const ImageRegion<VImageDimension>
>
> &largestPossibleRegion);
>
> It will also need a virtual helper function and do some dynamic casts, as
>
> well as the ugliness of some macro to create the templated object from a
>
> parameter.
>
> 2) Create a new ImageIORegionSplitter hierarchy, and duplicate the a bunch
>
> of code from ImageRegionSplitter. This will have a smoother interface:
>
> virtual ImageIORegionSpliter::Pointer
>
> ImageIOBase::GenerateWriteRegionSplitter(unsigned int
>
> numberOfStreamDivisions, const ImageIORegion<VImageDimension>
>
> largestPossibleRegion);
>
> Any thought and suggestions are welcome!
>
> Thanks,
>
> Brad
>
> On Dec 6, 2008, at 6:25 PM, Luis Ibanez wrote:
>
>
> Hi Brad,
>
>
> The functionality of Streaming is not fully implemented
>
> on the ImageFileWriter.
>
>
> The design proposal for implementing this feature is to move
>
> (copy/paste) part of the code that you find in the itkStreamingImageFilter
>
> (in Code/BasicFilters) to the
>
> itkImageFileWriter class.
>
> In particular, the proposal is to use the itkImageRegionSplitter
>
> inside the ImageFileWriter.
>
> During the design discussions, one of the issues that came up
>
> is that only the specific ImageIO classes are qualified to know
>
> what would be the appropriate way to split the data in order to
>
> match the type of blocks that the output image file format can
>
> manage (e.g. DICOM can only stream slices).
>
>
> Therefore, the suggested design is that the ImageFileWriter
>
> will ask the ImageIO class to provide a specific instance of
>
> an itkImageRegionSplitter, and then it will use it in a for
>
> loop for generating sub-regions of the image, and request
>
> them from the preceding image filters.
>
>
> --
>
> Until this is implemented, then your best options, is to do
>
> just what you are doing: using the itkStreamingImageFilter
>
> just before the ImageFileWriter. The drawback of course is
>
> that you have to whole in memory the full-size output image
>
> before being able to write it to disk.
>
>
> Please let us know if you have further questions,
>
>
> Thanks
>
>
> Luis
>
>
> -------------------------
>
> Bradley Lowekamp wrote:
>
> Greetings all!
>
> I am trying to stream my data set. I see that MetaIO supports streaming for
>
> reading and writing. But I don't seem to be able to use it for writing. I am
>
> trying to stream my 13GB data set from a ImageFileReader through a
>
> ShrinkImageFilter then write is out with a ImageFileWriter. If I replace the
>
> writer with a StreamingImageFilter it works great, the reader streams and
>
> everything.
>
> To get the writer to stream (or not) I am doing this:
>
> shrinker->UpdateOutputInformation();
>
> RGBVolumeType::RegionType outputRegion =
>
> shrinker->GetOutput()->GetLargestPossibleRegion();
>
> itk::ImageRegionSplitter<3>::Pointer splitter =
>
> itk::ImageRegionSplitter<3>::New();
>
> numberOfSplits = splitter->GetNumberOfSplits(outputRegion, numberOfSplits);
>
> writer->SetFileName( outputFilePN.GetPathName());
>
> writer->SetInput(shrinker->GetOutput());
>
> for(unsigned int i = 0; i < numberOfSplits; ++i) {
>
> RGBVolumeType::RegionType streamRegion = splitter->GetSplit(i,
>
> numberOfSplits, outputRegion);
>
> ioRegion = streamRegion; // sudo code
>
> writer->SetIORegion(ioRegion);
>
> writer->Update();
>
> }
>
> Is this the correct approach? Does anyone have an example of streaming
>
> writing? I think I am going to dig though the ImageFileWriter now to see
>
> what are the updates going on in the pipeline execution. I still a bit fuzzy
>
> on these details, so may miss something.
>
> Please let me know of any suggestions.
>
> Thanks,
>
> Brad
>
> ========================================================
>
> Bradley Lowekamp Lockheed Martin Contractor for
>
> Office of High Performance Computing and Communications
>
> National Library of Medicine blowekamp at mail.nih.gov
>
> <mailto:blowekamp at mail.nih.gov>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
>
> Insight-users mailing list
>
> Insight-users at itk.org
>
> http://www.itk.org/mailman/listinfo/insight-users
>
> ========================================================
>
> Bradley Lowekamp
>
> Lockheed Martin Contractor for
>
> Office of High Performance Computing and Communications
>
> National Library of Medicine
>
> blowekamp at mail.nih.gov
>
> _______________________________________________
>
> Insight-developers mailing list
>
> Insight-developers at itk.org
>
> http://www.itk.org/mailman/listinfo/insight-developers
>
> ========================================================
>
> Bradley Lowekamp
>
> Lockheed Martin Contractor for
>
> Office of High Performance Computing and Communications
>
> National Library of Medicine
>
> blowekamp at mail.nih.gov
>
>
> _______________________________________________
>
> Insight-developers mailing list
>
> Insight-developers at itk.org
>
> http://www.itk.org/mailman/listinfo/insight-developers
>
>
>
> ========================================================
>
> Bradley Lowekamp
>
> Lockheed Martin Contractor for
>
> Office of High Performance Computing and Communications
>
> National Library of Medicine
>
> blowekamp at mail.nih.gov
>
>
>
> ========================================================
>
> Bradley Lowekamp
>
> Lockheed Martin Contractor for
>
> Office of High Performance Computing and Communications
>
> National Library of Medicine
>
> blowekamp at mail.nih.gov
>
>
More information about the Insight-developers
mailing list