[Insight-developers] [ITK Community] HDF5 writer streaming

[Matt McCormick]

Hi David,

The Meta Image IO can currently stream write, so it may be helpful to
look at it, too.

Hope this helps,
Matt

On Mon, Dec 30, 2013 at 8:05 AM, Bradley Lowekamp
<blowekamp at mail.nih.gov> wrote:
> This article  may be of some use too:
> http://www.insight-journal.org/browse/publication/729
> On Dec 30, 2013, at 1:40 AM, David Froger <david.froger at inria.fr> wrote:
>
>> Hi all,
>>
>> I would like to add writer streaming support to the HDF5 module. Writer
>> streaming would includes:
>>
>> 1) Write a whole array in a new HDF5 file by iteration on subregion writting.
>> For example: write NZ times an array of dimension (NY*NX) in a new HDF5 file
>> that will finally store an array of dimensions (NZ*NY*NX).
>>
>> 2) Paste (override) a subregion in an existing HDF5 file. (This require to
>> open the file in a different mode).
>>
>> I know how to do it in HDF5, I've looked at the source code of the HDF5 ITK
>> module, and have some questions on how to patch the HDF5 module.
>>
>> To support reading and writing subregion, HDF5 manipulates simple arrays
>> defining the start and the size of the region to read/write. These arrays are
>> 'HDFSize' and 'offset' in the method HDF5ImageIO::SetupStreaming of the file
>> itkHDF5ImageIO.cxx, and their values are set according to the size and index
>> of the region 'this->GetIORegion()':
>>
>>   void
>>   HDF5ImageIO
>>   ::SetupStreaming(H5::DataSpace *imageSpace, H5::DataSpace *slabSpace)
>>   {
>>     ImageIORegion            regionToRead = this->GetIORegion();
>>     ImageIORegion::SizeType  size = regionToRead.GetSize();
>>     ImageIORegion::IndexType start = regionToRead.GetIndex();
>>     //
>>     int numComponents = this->GetNumberOfComponents();
>>
>>     const int HDFDim(this->GetNumberOfDimensions() +
>>                      (numComponents > 1 ? 1 : 0));
>>     hsize_t *offset = new hsize_t[HDFDim];
>>     hsize_t *HDFSize = new hsize_t[HDFDim];
>>     int limit;
>>     //
>>     // fastest moving dimension is intra-voxel
>>     // index
>>     if(numComponents > 1)
>>       {
>>       limit = HDFDim - 1;
>>       offset[limit] = 0;
>>       HDFSize[limit] = numComponents;
>>       }
>>     else
>>       {
>>       limit = HDFDim;
>>       }
>>     for(int i=0; i < limit; i++)
>>       {
>>       offset[limit - i - 1] = start[i];
>>       HDFSize[limit - i - 1] = size[i];
>>       }
>>     slabSpace->setExtentSimple(HDFDim,HDFSize);
>>     imageSpace->selectHyperslab(H5S_SELECT_SET,HDFSize,offset);
>>     delete[] HDFSize;
>>     delete[] offset;
>>   }
>>
>> When reading a file, 'this->GetIORegion()' can be modified by calling
>>   reader->GetOutput()->SetRequestedRegion(requestedRegion);
>> which enables input streaming.
>>
>> But how to modifiy 'this->GetIORegion()' when writing in a file, for case 1)
>> and 2)?
>>
>> Which example / source code can I look to get started with writer streaming?
>>
>> Thanks for your help!
>>
>> David
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Kitware offers ITK Training Courses, for more information visit:
>> http://kitware.com/products/protraining.php
>>
>> Please keep messages on-topic and check the ITK FAQ at:
>> http://www.itk.org/Wiki/ITK_FAQ
>>
>> Follow this link to subscribe/unsubscribe:
>> http://www.itk.org/mailman/listinfo/insight-developers
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Kitware offers ITK Training Courses, for more information visit:
> http://kitware.com/products/protraining.php
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.itk.org/mailman/listinfo/insight-developers
> _______________________________________________
> Community mailing list
> Community at itk.org
> http://public.kitware.com/cgi-bin/mailman/listinfo/community