[vtk-developers] Speeding up `import vtk`

[Matt McCormick]

Hi,

Yes, the lazy loading has been working very well for ITK. In addition
to the previous links to the implementation, a few other things to be
aware of are

- The dependencies are stored via CMake's configure_file on .py files
- There is an option to disable lazy loading in the itkConfig module,
which can be imported before "itk"
- An import callback can be configured to monitor / debug the process

VTK_ENABLE_KITS may actually be a little bit slower in the case of
lazy loading because more unused symbols have to be loaded, but that
performance assumption needs to be tested.

HTH,
Matt

On Thu, Sep 21, 2017 at 7:10 PM, Jean-Christophe Fillion-Robin
<jchris.fillionr at kitware.com> wrote:
> Hi,
>
> Lazy loading implemented in ITK python has been working well. Since we have
> knowledge of the VTK dependency graph, I suggest to use a similar approach.
>
> Also building VTK with VTK_ENABLE_KITS enabled greatly reduces the number of
> libraries and could avoid to implement such optimization.
>
> Matt (cc'ed) should be able to provide feedback.
>
> Hth
> Jc
>
> On Wed, Sep 20, 2017 at 9:18 AM, David Gobbi <david.gobbi at gmail.com> wrote:
>>
>> Hi Ben,
>>
>> I like Matt's comments on lazy-loading, it would allow us to keep
>> backwards compatibility while speeding up load times. Python provides a way
>> to override getattr() for modules:
>> https://stackoverflow.com/questions/2447353/getattr-on-a-module
>>
>>  - David
>>
>> On Tue, Sep 19, 2017 at 8:29 AM, Ben Boeckel <ben.boeckel at kitware.com>
>> wrote:
>>>
>>> [ Yes, this is a necro on this thread, but I'd like to see this happen. ]
>>>
>>> On Thu, Sep 08, 2016 at 11:14:21 -0600, David Gobbi wrote:
>>> > As usual, I'm a stickler for backwards compatibility and would rather
>>> > go in
>>> > the other direction,
>>>
>>> How about fixing `vtk` for VTK9? I don't like having it be a separate
>>> module because that means we have to ship things twice and the only
>>> difference is the `__init__.py` file.
>>>
>>> Currently `import vtk` (the obvious thing to do) is basically like doing
>>> a `find_package(VTK REQUIRED)` that doesn't support `COMPONENTS`. You
>>> get everything built, but you also have no idea if what you need is
>>> actually present until you get your `AttributeError`.
>>>
>>> I'd rather see scripts explicitly request `vtk.FiltersParallel` (or
>>> similar) if they need it rather than just hoping it dumped the right
>>> things into the global namespace. If we want to keep having this
>>> pattern, ParaView's `paraview.simple` module provides everything needed
>>> to "hit the ground running" (though I don't see it as a long-term
>>> maintainable way to use VTK or ParaView via Python). VTK could also have
>>> `vtk.simple` which does this. The problem with having it at the root
>>> level is that it denies any advanced usage of the module which does
>>> request exactly what is required.
>>>
>>> With all the other big changes coming in VTK9, I think it would be a
>>> good time to tackle this.
>>>
>>> --Ben
>>>
>>> P.S. For those curious, this came up because we recently had one module
>>> have more "load commands" than the loader on macOS supports
>>> (vtkPVAnimation) which triggered an `ImportError`. Because `import vtk`
>>> loads all modules, this meant that any loading of `import vtk` was
>>> broken just because this one was, even tests which don't care about it
>>> at all.
>>
>>
>>
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/vtk-developers
>>
>>
>
>
>
> --
> +1 919 869 8849