[vtkusers] Streaming pipeline with parts of big volume in multiple views

[Elvis Stansvik]

The attached sketch gives a better idea of the views I'm after.

Elvis

2016-02-07 11:41 GMT+01:00 Elvis Stansvik <elvis.stansvik at orexplore.com>:

> Hi all,
>
> As I tried to explain in a previous mail
>
>    http://markmail.org/message/mfp2npib5nqeeuux
>
> I have a large amount of volume data, and I need to render only a small
> part of it.
>
> I've been trying to find information on the streaming visualization
> pipeline that VTK supports, but it's been a bit hard to find examples
> similar what I want to do.
>
> My questions right now are:
>
> 1) Is there some file format for volume data which is supported by VTK
> with a reader that is "intelligent" enough to only read the parts of the
> file that is requested by downstream consumers into main memory? That is,
> is there something I could use so that my use case would work "out of the
> box". If there is, I'm prepared to make sure the output from our analysis
> is in this format, as it would save me from writing a custom reader.
>
> 2) If there isn't, does anyone know of an example reader somewhere that
> has this intelligence? I'm prepared to write my own reader, but would
> really like a good example of how such a reader can be written.
>
> 3) I assume that all parts of the pipeline I set up must be capable of
> working with partial data (extents in my case) for this to work? That is, I
> obviously can't have something in the pipeline which is a "bad citizen" and
> spoils it, causing the pipeline to request all the data. Is there a list of
> which classes that supports this? Or are all filters et.c. in VTK
> well-written in the sense that they will only request the minimal amount of
> data they need, based on downstream requirements?
>
> 4) If I set the volume of interest for a volume using a vtkExtractVOI,
> will the information about which piece of the volume is required reach the
> source (reader) at the start of the pipeline?
>
> 5) I was planning on having a pipeline that looks roughly like
>
>     Reader ----> Extract VOI ----> Do stuff ---> View1
>          \
>           +------> Extract VOI ----> Do other stuff ---> View2
>
> where each of the two views show a different part of the full volume. One
> of the views is scrollable, and the idea is that when the user scrolls that
> view, the VOI is updated and that branch of the pipeline is re-executed.
> Only the required data should be loaded from disk into main memory when
> this happens. Is this kind of setup possible? Or will I have to use two
> completely separate pipelines?
>
> 6) If it turns out there are no built-in file formats where the reader
> supports loading parts of the data from the file on disk based on
> downstream requirements, I'm leaning towards making our own HDF5-based
> format + reader (I believe the full volume will be around 50-100 GB of
> data), since I think (but I'm not sure) that it's possible to only read
> parts of a 3D array from a HDF5 file using h5py. Does anyone else have a
> good suggestion for a file format, or have you been in a similar situation?
>
> Very grateful for any advice.
>
> Best regards,
> Elvis
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtkusers/attachments/20160207/9b699011/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: core_views.png
Type: image/png
Size: 205373 bytes
Desc: not available
URL: <http://public.kitware.com/pipermail/vtkusers/attachments/20160207/9b699011/attachment-0001.png>