[vtk-developers] Memory-mapped files in VTK

[Burlen Loring]

Hi David,

I've used mmap with VTK in C++ for the SC11 100GB network demo.

http://dl.acm.org/citation.cfm?id=2287004
https://github.com/burlen/SC11-100GB-Demo

This use was very ad-hoc and we wanted the data mapped for the life of 
the app so there were special memory management issues.

patch(http://review.source.kitware.com/#/c/14072/) would have handled 
situations such as this through a callback invoked when the data array 
is finished with the pointer. However, that patch was rejected for being 
overkill. Instead Berk suggested that I simply use the delete event 
observer, which will always result in data being released but perhaps 
not as quickly as with the callback or the method that you proposed(eg 
nothing happens if the array is resized). This approach has worked well 
for me. I wonder if using the delete event observer would work well in 
your case?

Burlen


On 05/16/2014 10:21 AM, David Gobbi wrote:
> Hi All,
>
> I've used memory mapped files with VTK in the past, using the python
> "mmap" object to create the map and then passing the resulting pointer
> to VTK via the SetVoidArray() method.  For large files (several GB) I've
> found this to be many times faster than e.g. vtkImageReader2.
>
> My question: has anyone used mmap with VTK in C++?  Specifically,
> what I'm interested in doing is adding a new "delete method" to the
> SetVoidPointer() method in vtkDataArrayTemplate.h: it already has
> "delete methods" for "malloc" and "new", so the idea is to add another
> delete method for "mmap" that will call munmap on the pointer.
>
>    David
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://www.vtk.org/mailman/listinfo/vtk-developers
>