[vtk-developers] Python 3.7 behavior change, GIL, and threading in C/C++ with Python

Marcus D. Hanwell marcus.hanwell at kitware.com
Mon Nov 26 11:59:20 EST 2018


I have seen several fixes for Python 3.7, but have been tracking down why
Tomviz deadlocks on when using vtkPythonScopeGilEnsurer, and it appears to
be related to the Python specific flags we are using and a behavior change
in Python 3.7. This may mainly be a question for David Gobbi and/or
Utkarsh. but I send it to the list so that others have a chance to see it,
and it gets indexed should others hit this issue.

to OFF. This means that when vtkPythonInterpreter::Initialize is called
that the following code will be called:

    int threadInit = PyEval_ThreadsInitialized();
    PyEval_InitThreads(); // safe to call this multiple time
    if (!threadInit)
      PyEval_SaveThread(); // release GIL

This was effectively always calling PyEval_SaveThread() before Python 3.7.
but this is now never being called as starting with Python 3.7
Py_Initialize will initialize the GIL, see
https://docs.python.org/3/c-api/init.html#c.PyEval_InitThreads for details.

It looks like this check was done to ensure that PyEval_SaveThread would
only be called once, now that PyEval_InitThreads doesn't need to be called,
moving PyEval_SaveThread outside of the conditional gets rid of the
deadlock for me. It looks like something needs to be changed for 3.7+, but
I am not clear on how necessary this logic is.

This reminds me that VTK_PYTHON_FULL_THREADSAFE is used in VTK, but the
CMake option is in ParaView only as far as I can see. Should that be moved
down into VTK, it seems odd.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://public.kitware.com/pipermail/vtk-developers/attachments/20181126/83bdc039/attachment-0001.html>

More information about the vtk-developers mailing list