[vtk-developers] Patch to vtkWrapPython.c to make wrapped Python code GIL-friendly

[Prabhu Ramachandran]

>>>>> "Charl" == Charl P Botha <cpbotha at gmail.com> writes:

    Charl> On 6/17/05, David Gobbi <dgobbi at atamai.com> wrote:
    >> On the other point about observers being called from a
    >> different thread than the one that set the observer, it seems
    >> that you need to maintain a data structure that keeps track of
    >> all the python thread structs and you need some way of figuring
    >> out which one to use when an observer is called.  It definitely
    >> sounds doable to me.

    Charl> Is this allowed in C++?  I.e. if I'd create an observer in
    Charl> one thread, and then add it to a VTK object in another
    Charl> thread, does VTK guarantee that this will work?

I'm sorry but I don't see how this is an issue with C++ or VTK.  VTK
sees the vtkPythonCommand object and that in turn routes the call
through Python.  So what is the problem with VTK or C++ here?

    Charl> With regards to the data structure that you suggest: the
    Charl> question is how critical it is to get an up-to-date
    Charl> PyThreadStruct.  If this needs to be kept completely up to
    Charl> date, then it's not as straight-forward: by the time the
    Charl> long running C code runs, getting the PyThreadStates of all
    Charl> its various observers that could be running in different
    Charl> threads is very difficult.  If the PyThreadState does not
    Charl> have to be kept completely up to date, then my solution of
    Charl> doing it in the constructor of the vtkPythonCommand is not
    Charl> such a bad idea: this means that the vtkPythonCommand will
    Charl> always be executed in the thread that it was created.  I'm
    Charl> with Prabhu on this one: it does seem that the ThreadState
    Charl> needs to be up to date.

All releasing the GIL does is to allow *other* Python threads to run.
The *current* thread will be executing the VTK method.  So if you save
the state of the thread before the VTK method call and restore that
state once its done, then that particular thread state should not have
changed at all.  This is assuming no observers were called but let us
ignore that for the time being.  

I think you only need to worry about how to deal with observers,
running in the same or other threads, and saving the state just before
you execute the VTK method.  There is no issue with the VTK method
taking long to run and therefore the thread state changing.  I think
both issues are solved with the vtkThreadManager. :)

    Charl> However, one could also say that by definition, all
    Charl> observer objects will be executed in the same thread as the
    Charl> object that they are observing.  I would then somehow store
    Charl> a PyThreadState whenever the GIL is given up and allow the
    Charl> vtkPythonCommand to get at this stored value.  IMO, this is
    Charl> the simplest and most predictable solution, and thus also
    Charl> the one I prefer.

I would think the vtkThreadManager would be just as easy to write and
solves all the problems.  There is only one GIL, therefore only one
*active* thread for Python at any point of time.  So you need to keep
one stack of thread states and pop/append states into it.  When a VTK
method call embedded inside an observer callback needs the lock, it
releases the earlier lock, acquires it, does its thing, releases the
lock and re-acquires the lock and sets the original state.

cheers,
prabhu