[Insight-developers] [ITK Community] HDF5 writer streaming

[David Froger]

Thank you Bradley and Matt, I look at both, that's helping!

David

Quoting Matt McCormick (2014-01-05 05:41:50)
> Hi David,
> 
> The Meta Image IO can currently stream write, so it may be helpful to
> look at it, too.
> 
> Hope this helps,
> Matt
> 
> On Mon, Dec 30, 2013 at 8:05 AM, Bradley Lowekamp
> <blowekamp at mail.nih.gov> wrote:
> > This article  may be of some use too:
> > http://www.insight-journal.org/browse/publication/729
> > On Dec 30, 2013, at 1:40 AM, David Froger <david.froger at inria.fr> wrote:
> >
> >> Hi all,
> >>
> >> I would like to add writer streaming support to the HDF5 module. Writer
> >> streaming would includes:
> >>
> >> 1) Write a whole array in a new HDF5 file by iteration on subregion writting.
> >> For example: write NZ times an array of dimension (NY*NX) in a new HDF5 file
> >> that will finally store an array of dimensions (NZ*NY*NX).
> >>
> >> 2) Paste (override) a subregion in an existing HDF5 file. (This require to
> >> open the file in a different mode).
> >>
> >> I know how to do it in HDF5, I've looked at the source code of the HDF5 ITK
> >> module, and have some questions on how to patch the HDF5 module.
> >>
> >> To support reading and writing subregion, HDF5 manipulates simple arrays
> >> defining the start and the size of the region to read/write. These arrays are
> >> 'HDFSize' and 'offset' in the method HDF5ImageIO::SetupStreaming of the file
> >> itkHDF5ImageIO.cxx, and their values are set according to the size and index
> >> of the region 'this->GetIORegion()':
> >>
> >>   void
> >>   HDF5ImageIO
> >>   ::SetupStreaming(H5::DataSpace *imageSpace, H5::DataSpace *slabSpace)
> >>   {
> >>     ImageIORegion            regionToRead = this->GetIORegion();
> >>     ImageIORegion::SizeType  size = regionToRead.GetSize();
> >>     ImageIORegion::IndexType start = regionToRead.GetIndex();
> >>     //
> >>     int numComponents = this->GetNumberOfComponents();
> >>
> >>     const int HDFDim(this->GetNumberOfDimensions() +
> >>                      (numComponents > 1 ? 1 : 0));
> >>     hsize_t *offset = new hsize_t[HDFDim];
> >>     hsize_t *HDFSize = new hsize_t[HDFDim];
> >>     int limit;
> >>     //
> >>     // fastest moving dimension is intra-voxel
> >>     // index
> >>     if(numComponents > 1)
> >>       {
> >>       limit = HDFDim - 1;
> >>       offset[limit] = 0;
> >>       HDFSize[limit] = numComponents;
> >>       }
> >>     else
> >>       {
> >>       limit = HDFDim;
> >>       }
> >>     for(int i=0; i < limit; i++)
> >>       {
> >>       offset[limit - i - 1] = start[i];
> >>       HDFSize[limit - i - 1] = size[i];
> >>       }
> >>     slabSpace->setExtentSimple(HDFDim,HDFSize);
> >>     imageSpace->selectHyperslab(H5S_SELECT_SET,HDFSize,offset);
> >>     delete[] HDFSize;
> >>     delete[] offset;
> >>   }
> >>
> >> When reading a file, 'this->GetIORegion()' can be modified by calling
> >>   reader->GetOutput()->SetRequestedRegion(requestedRegion);
> >> which enables input streaming.
> >>
> >> But how to modifiy 'this->GetIORegion()' when writing in a file, for case 1)
> >> and 2)?
> >>
> >> Which example / source code can I look to get started with writer streaming?
> >>
> >> Thanks for your help!
> >>
> >> David
> >> _______________________________________________
> >> Powered by www.kitware.com
> >>
> >> Visit other Kitware open-source projects at
> >> http://www.kitware.com/opensource/opensource.html
> >>
> >> Kitware offers ITK Training Courses, for more information visit:
> >> http://kitware.com/products/protraining.php
> >>
> >> Please keep messages on-topic and check the ITK FAQ at:
> >> http://www.itk.org/Wiki/ITK_FAQ
> >>
> >> Follow this link to subscribe/unsubscribe:
> >> http://www.itk.org/mailman/listinfo/insight-developers
> >
> > _______________________________________________
> > Powered by www.kitware.com
> >
> > Visit other Kitware open-source projects at
> > http://www.kitware.com/opensource/opensource.html
> >
> > Kitware offers ITK Training Courses, for more information visit:
> > http://kitware.com/products/protraining.php
> >
> > Please keep messages on-topic and check the ITK FAQ at:
> > http://www.itk.org/Wiki/ITK_FAQ
> >
> > Follow this link to subscribe/unsubscribe:
> > http://www.itk.org/mailman/listinfo/insight-developers
> > _______________________________________________
> > Community mailing list
> > Community at itk.org
> > http://public.kitware.com/cgi-bin/mailman/listinfo/community