[vtk-developers] Memory-mapped files in VTK

[David Gobbi]

Hi Burlen,

Thanks for the links.  I guess that I'm looking for deeper integration
(though less general) than a callback.  The idea is to have something
like a vtkMemoryMap class to take care of all the platform-specific
details of creating/deleting memory maps, and to have vtkAbstractArray
be aware of this class and use it to delete mmaps when appropriate.

  David

On Fri, May 16, 2014 at 11:56 AM, Burlen Loring <burlen.loring at gmail.com> wrote:
> meant to say "no special memory management issues"
> and (in case you actually would look at the code) mmap use is here:
> https://github.com/burlen/SC11-100GB-Demo/blob/master/SC11DemoReader.h
>
> On 05/16/2014 10:50 AM, Burlen Loring wrote:
>
> Hi David,
>
> I've used mmap with VTK in C++ for the SC11 100GB network demo.
>
> http://dl.acm.org/citation.cfm?id=2287004
> https://github.com/burlen/SC11-100GB-Demo
>
> This use was very ad-hoc and we wanted the data mapped for the life of the
> app so there were special memory management issues.
>
> patch(http://review.source.kitware.com/#/c/14072/) would have handled
> situations such as this through a callback invoked when the data array is
> finished with the pointer. However, that patch was rejected for being
> overkill. Instead Berk suggested that I simply use the delete event
> observer, which will always result in data being released but perhaps not as
> quickly as with the callback or the method that you proposed(eg nothing
> happens if the array is resized). This approach has worked well for me. I
> wonder if using the delete event observer would work well in your case?
>
> Burlen
>
>
> On 05/16/2014 10:21 AM, David Gobbi wrote:
>
> Hi All,
>
> I've used memory mapped files with VTK in the past, using the python
> "mmap" object to create the map and then passing the resulting pointer
> to VTK via the SetVoidArray() method.  For large files (several GB) I've
> found this to be many times faster than e.g. vtkImageReader2.
>
> My question: has anyone used mmap with VTK in C++?  Specifically,
> what I'm interested in doing is adding a new "delete method" to the
> SetVoidPointer() method in vtkDataArrayTemplate.h: it already has
> "delete methods" for "malloc" and "new", so the idea is to add another
> delete method for "mmap" that will call munmap on the pointer.
>
>    David