[Paraview] Using JOIN to load images from Many HDF files.

[Marcus D. Hanwell]

John,

Thanks for the additional information, this is really useful and
certainly helps us as we look at data coming from experimental domains
such as this. I will take a look at the links provided, and am very
interested in hearing more as you work with facility users to make use
of ParaView in this area.

I did some work on improving the TIFF reader (already merged), and
added support for 32 bit floats among other things. Were you thinking
of this, http://www.aps.anl.gov/DataExchange/, for the tomography
data? I hadn't come across it, but it is great to see it so
well-defined with example code.

Thanks,

Marcus

On Fri, Jul 11, 2014 at 5:40 PM, John Hammonds <JPHammonds at anl.gov> wrote:
> Marcus,
>
> The HDF data that I am working with is currently collected using the NeXus
> data format (http://www.nexusformat.org/).  Some of the data that is
> collected has all of the data in one big file and others have the data in
> separate files.  For the case that all of the data is in one big file, XDMF
> actually works pretty smoothly once you get over the initial hurdles.  For
> the case that the data is in different files, I had the issues that I
> described.  Note that I have also looked the last couple of days on loading
> Tiff images (for another beamline here www.aps.anl.gov) using Binary instead
> of HDF. This went pretty smoothly except that the XDMF pages do not mention
> that you can use Short as the NumberType for 16 bit data. Eventually I tried
> it and it fixed my loading issues.
>
> Note that we also have some Tomography beamlines here.  Their data is
> captured in a similar but different HDF format (can't see to find the link
> at the moment).
>
> John
>
> On 7/11/2014 2:42 PM, Marcus D. Hanwell wrote:
>>
>> On Fri, Jul 11, 2014 at 10:51 AM, John Hammonds <JPHammonds at anl.gov>
>> wrote:
>>>
>>> I tried posting this to the XDMF mailing list and have received no
>>> response.
>>> Hoping someone here can help...
>>>
>>> I have a number of images(a few hundred) , each collected in an
>>> individual
>>> HDF file, that I am trying to read into Paraview using a XDMF file.  I
>>> have
>>> found that I can use JOIN to do this but it seems that I need to nest the
>>> JOINs.  The nesting seems to be due to some limit on the number of things
>>> that can be referenced in the JOIN.  I seem to be limited to ten items in
>>> the join list.  All of this has me building some odd Join files due to
>>> the
>>> way things are joined at each level.  I am wondering if I am missing
>>> something that would make this a lot simpler.  Esp since I seem to have
>>> trouble when large sets are not the same size.  I am including a working
>>> XDMF file an example of one of the HDF files for reference.
>>>
>> How is your data produced? We are developing a custom ParaView
>> application, and as part of that we are adding some new readers to VTK
>> to support S/TEM tomography using an HDF5-based format. The spec is
>> here,
>>
>> http://emdatasets.lbl.gov/spec/
>>
>> It would be great to understand your needs, and whether some of this
>> development work could be tweaked in order to address a wider range of
>> experimental data that might help your user community. If XDMF is one
>> path, then we should also look more closely at supporting that too.
>>
>> Thanks,
>>
>> Marcus
>
>
> --
> --------------------------------------------
> John Hammonds
> Software Services Group
> Advanced Photon Source
> Argonne National Laboratory
> JPHammonds at anl.gov
>