[vtk-developers] Streaming slow, requires a lot of memory

[Brad King]

tom fogal wrote:
> In both MayaVi and a custom C++ app I've developed, the datasets take a
> prohibitive amount of memory (and thus time) to work with.
> 
> I was under the impression that once I had a reader/format that
> supported streaming, the maximum amount of memory that would be
> required for processing would be the size of a piece plus some caching
> overhead. It seems as if the entire dataset is being read in regardless
> of what portions are to be used.

It is up to the consumer at the end of a pipeline to ask for the 
individual pieces to get streaming to work.  If it just asks for 
everything then even a streaming reader will read everything.

Also the pipeline by default keeps a copy of the output of each filter, 
so you can have several copies of the data lying around.  You can use 
the SetReleaseDataFlag method to tell filters to release their input 
data after producing their output to keep memory usage down at the 
expense of computation time.

-Brad