[Insight-developers] Streaming with ImageWriter and IORegion
Stephen Aylward
Stephen.Aylward at kitware.com
Mon Dec 22 11:49:12 EST 2008
Hi,
Quick note...we are using SetIORegion for streaming writing in a commercial
product.
Streaming reading and writing is working fine when .mhd/.mha files are used.
We need to maintain backward compatibility.
Thanks,
s
On Mon, Dec 22, 2008 at 10:01 AM, Bill Lorensen <bill.lorensen at gmail.com>wrote:
> Brad,
>
> Sounds great.
>
> Bill
>
> On Mon, Dec 22, 2008 at 9:08 AM, Bradley Lowekamp
> <blowekamp at mail.nih.gov> wrote:
> > Bill,
> > I'll implement this method how I think is should work. That is I'll make
> is
> > so that it requests the ioregion, up the pipeline and will tolerate if
> the
> > exact region is not supplied. This will maintain the backwards
> compatibility
> > of the interface, but the behavior may be a little different.
> > Then I will submit the changes into mantis so that they can be quickly
> > reviewed or what not.
> >
> > Brad
> > On Dec 19, 2008, at 6:48 PM, Bill Lorensen wrote:
> >
> > Brad,
> >
> > Really, we don't know what customers may be using this method. Maybe
> > even correctly in their own ImageIO class. Many customers write their
> > own ImageIO classes. For example, Slicer3 has two of its own ImageIO
> > classes. To be honest, I have not looked at how it's used in the
> > ImageFileWriter or current ImageIO classes. I'll take a look over the
> > next few days.
> >
> > But, as I said before, we released the code with these methods. Since
> > we made the mistake, the burden is on us to fix it, not our customers.
> >
> > Bill
> >
> > On Fri, Dec 19, 2008 at 5:05 PM, Bradley Lowekamp
> > <blowekamp at mail.nih.gov> wrote:
> >
> > Bill,
> >
> > This is a very problematic method; I have strong doubts that anyone is
> using
> >
> > it. It is unclear to me what the appropriate behavior should be. If it's
> >
> > current behavior must be maintained then, we should copy the current
> Write
> >
> > method, so that it can be called only when needed if this method has been
> >
> > set, Then depreciate the method so that it can be removed later.
> >
> > MetaImageIO is the only class that can support this method. Use it with
> any
> >
> > other ImageIO (I think) results in no image, or undefined behavior. I
> >
> > believe that this method was a start of implementing streaming that was
> >
> > never finished. For it to be used the input to ImageFileWrite must have
> >
> > already executed and generated this exact region or else the results
> would
> >
> > be wrong, and undefined. Some of these problems could be reduced if
> >
> > ImageFileWrite copies the regionIO to a buffer/cache if needed, then it
> >
> > would be up to the ImageIO to do what is asked or barf.
> >
> > I would argue that anyone using this method should not be using
> >
> > ImageFileWriter and should just use the ImageIO object directly, with
> it's
> >
> > current behavior, but that's not backwardly compatible.
> >
> > What's be best solution?
> >
> > Brad
> >
> > On Dec 19, 2008, at 3:59 PM, Bill Lorensen wrote:
> >
> > Brad,
> >
> > We cannot remove methods from a class that has been released.We should
> >
> > do what we can to avoid their improper use.
> >
> > Bill
> >
> > On Fri, Dec 19, 2008 at 3:17 PM, Bradley Lowekamp
> >
> > <blowekamp at mail.nih.gov> wrote:
> >
> > Hello,
> >
> > I have essentially completed these modifications. There is just more
> testing
> >
> > to write. All of the current non-IO streaming tests are passing (that
> should
> >
> > be). The only ImageIO class that was ready for streamed writing was
> MetaIO,
> >
> > so that is currently the only file type that is streaming.
> >
> > Option number 1, was chosen from bellow. It was the least amount of code,
> >
> > since it doesn't double the splitters.
> >
> > On the down side of these changes, I removed
> >
> > ImageFileWriter::Set/GetIORegion. These were low level methods which are
> not
> >
> > compatible with the ITK pipeline, and likely to cause segfaults and
> crashes
> >
> > if not used just right.
> >
> > What is the best way to contribute this code? IJ, attach the diff to
> Mantis?
> >
> > Just submit it to CVS, and hope CDash doesn't light up like a xmas tree?
> I
> >
> > think it's good quality, just may have missed some nuances of ITK or a
> >
> > compiler compatibility.
> >
> > I am excited to be able to process my 13GB RGB data set on my laptop :)
> >
> > Brad
> >
> > On Dec 18, 2008, at 12:23 PM, Bradley Lowekamp wrote:
> >
> > Hello,
> >
> > I am implementing streaming in ImageFileWriter, hoping that it can get on
> >
> > the fast track to making it into the repository. I have done most of the
> >
> > work in ImageFileWriter and now working out some difficulties with
> >
> > ImageIOBase. According to the design Luis wrote below, IOBase should have
> a
> >
> > virtual method to create a "region splitter". The difficulties with this
> >
> > lies in the differences between ImageIORegion and ImageRegion.
> Specifically
> >
> > ImageRegion is templated over dimension for use with Image, where
> >
> > ImageIORegion is not since ImageIO classes are not. The issues then comes
> >
> > from with ImageRegionSplitter<#> being templeted. So there becomes two
> >
> > choices I see:
> >
> > 1) Use ImageRegionSplitter in ImageIO, this is have the
> >
> > following declaration:
> >
> > template <unsigned int VImageDimension>
> >
> > typename ImageRegionSplitter<VImageDimension>::Pointer
> >
> > ImageIOBase::GenerateWriteRegionSplitter(unsigned int
> >
> > numberOfStreamDivisions, const ImageRegion<VImageDimension>
> >
> > &largestPossibleRegion);
> >
> > It will also need a virtual helper function and do some dynamic casts, as
> >
> > well as the ugliness of some macro to create the templated object from a
> >
> > parameter.
> >
> > 2) Create a new ImageIORegionSplitter hierarchy, and duplicate the a
> bunch
> >
> > of code from ImageRegionSplitter. This will have a smoother interface:
> >
> > virtual ImageIORegionSpliter::Pointer
> >
> > ImageIOBase::GenerateWriteRegionSplitter(unsigned int
> >
> > numberOfStreamDivisions, const ImageIORegion<VImageDimension>
> >
> > largestPossibleRegion);
> >
> > Any thought and suggestions are welcome!
> >
> > Thanks,
> >
> > Brad
> >
> > On Dec 6, 2008, at 6:25 PM, Luis Ibanez wrote:
> >
> >
> > Hi Brad,
> >
> >
> > The functionality of Streaming is not fully implemented
> >
> > on the ImageFileWriter.
> >
> >
> > The design proposal for implementing this feature is to move
> >
> > (copy/paste) part of the code that you find in the
> itkStreamingImageFilter
> >
> > (in Code/BasicFilters) to the
> >
> > itkImageFileWriter class.
> >
> > In particular, the proposal is to use the itkImageRegionSplitter
> >
> > inside the ImageFileWriter.
> >
> > During the design discussions, one of the issues that came up
> >
> > is that only the specific ImageIO classes are qualified to know
> >
> > what would be the appropriate way to split the data in order to
> >
> > match the type of blocks that the output image file format can
> >
> > manage (e.g. DICOM can only stream slices).
> >
> >
> > Therefore, the suggested design is that the ImageFileWriter
> >
> > will ask the ImageIO class to provide a specific instance of
> >
> > an itkImageRegionSplitter, and then it will use it in a for
> >
> > loop for generating sub-regions of the image, and request
> >
> > them from the preceding image filters.
> >
> >
> > --
> >
> > Until this is implemented, then your best options, is to do
> >
> > just what you are doing: using the itkStreamingImageFilter
> >
> > just before the ImageFileWriter. The drawback of course is
> >
> > that you have to whole in memory the full-size output image
> >
> > before being able to write it to disk.
> >
> >
> > Please let us know if you have further questions,
> >
> >
> > Thanks
> >
> >
> > Luis
> >
> >
> > -------------------------
> >
> > Bradley Lowekamp wrote:
> >
> > Greetings all!
> >
> > I am trying to stream my data set. I see that MetaIO supports streaming
> for
> >
> > reading and writing. But I don't seem to be able to use it for writing. I
> am
> >
> > trying to stream my 13GB data set from a ImageFileReader through a
> >
> > ShrinkImageFilter then write is out with a ImageFileWriter. If I replace
> the
> >
> > writer with a StreamingImageFilter it works great, the reader streams and
> >
> > everything.
> >
> > To get the writer to stream (or not) I am doing this:
> >
> > shrinker->UpdateOutputInformation();
> >
> > RGBVolumeType::RegionType outputRegion =
> >
> > shrinker->GetOutput()->GetLargestPossibleRegion();
> >
> > itk::ImageRegionSplitter<3>::Pointer splitter =
> >
> > itk::ImageRegionSplitter<3>::New();
> >
> > numberOfSplits = splitter->GetNumberOfSplits(outputRegion,
> numberOfSplits);
> >
> > writer->SetFileName( outputFilePN.GetPathName());
> >
> > writer->SetInput(shrinker->GetOutput());
> >
> > for(unsigned int i = 0; i < numberOfSplits; ++i) {
> >
> > RGBVolumeType::RegionType streamRegion = splitter->GetSplit(i,
> >
> > numberOfSplits, outputRegion);
> >
> > ioRegion = streamRegion; // sudo code
> >
> > writer->SetIORegion(ioRegion);
> >
> > writer->Update();
> >
> > }
> >
> > Is this the correct approach? Does anyone have an example of streaming
> >
> > writing? I think I am going to dig though the ImageFileWriter now to see
> >
> > what are the updates going on in the pipeline execution. I still a bit
> fuzzy
> >
> > on these details, so may miss something.
> >
> > Please let me know of any suggestions.
> >
> > Thanks,
> >
> > Brad
> >
> > ========================================================
> >
> > Bradley Lowekamp Lockheed Martin Contractor for
> >
> > Office of High Performance Computing and Communications
> >
> > National Library of Medicine blowekamp at mail.nih.gov
> >
> > <mailto:blowekamp at mail.nih.gov>
> >
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> >
> > Insight-users mailing list
> >
> > Insight-users at itk.org
> >
> > http://www.itk.org/mailman/listinfo/insight-users
> >
> > ========================================================
> >
> > Bradley Lowekamp
> >
> > Lockheed Martin Contractor for
> >
> > Office of High Performance Computing and Communications
> >
> > National Library of Medicine
> >
> > blowekamp at mail.nih.gov
> >
> > _______________________________________________
> >
> > Insight-developers mailing list
> >
> > Insight-developers at itk.org
> >
> > http://www.itk.org/mailman/listinfo/insight-developers
> >
> > ========================================================
> >
> > Bradley Lowekamp
> >
> > Lockheed Martin Contractor for
> >
> > Office of High Performance Computing and Communications
> >
> > National Library of Medicine
> >
> > blowekamp at mail.nih.gov
> >
> >
> > _______________________________________________
> >
> > Insight-developers mailing list
> >
> > Insight-developers at itk.org
> >
> > http://www.itk.org/mailman/listinfo/insight-developers
> >
> >
> >
> > ========================================================
> >
> > Bradley Lowekamp
> >
> > Lockheed Martin Contractor for
> >
> > Office of High Performance Computing and Communications
> >
> > National Library of Medicine
> >
> > blowekamp at mail.nih.gov
> >
> >
> >
> > ========================================================
> >
> > Bradley Lowekamp
> >
> > Lockheed Martin Contractor for
> >
> > Office of High Performance Computing and Communications
> >
> > National Library of Medicine
> >
> > blowekamp at mail.nih.gov
> >
> >
> _______________________________________________
> Insight-developers mailing list
> Insight-developers at itk.org
> http://www.itk.org/mailman/listinfo/insight-developers
>
--
Stephen R. Aylward, Ph.D.
Chief Medical Scientist
Kitware, Inc. - North Carolina Office
http://www.kitware.com
(518) 371-3971 x300
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.itk.org/mailman/private/insight-developers/attachments/20081222/358901af/attachment.htm>
More information about the Insight-developers
mailing list