[vtkusers] VTK 7.0.0 + Python: GIL released?

[Paul Melis]

Good to know, thanks! 
Paul 


From: "Mathieu Westphal" <mathieu.westphal at kitware.com> 
To: "Paul Melis" <paul.melis at surfsara.nl> 
Cc: "vtk" <vtkusers at vtk.org> 
Sent: Thursday, 8 September, 2016 14:30:23 
Subject: Re: [vtkusers] VTK 7.0.0 + Python: GIL released? 

Hi 

Sorry, I did not see you were only on VTK. 
The option is exposed in ParaView cmake option, the same way VTK_NO_PYTHON_THREAD is. 
But you should be able to define the option manually when configuring VTK. 

-DVTK_PYTHON_FULL_THREADSAFE:BOOL="ON" 

Regards, 

Mathieu Westphal 

On Thu, Sep 8, 2016 at 2:11 PM, Paul Melis < paul.melis at surfsara.nl > wrote: 



Great! But the VTK_PYTHON_FULL_THREADSAFE option doesn't seem to exist at the CMake level. With a git update of a minute ago: 

/data/c/vtk-git$ find . -name CMakeLists.txt | xargs -iX grep -Hs VTK_PYTHON_FULL_THREADSAFE X 
<empty> 

/data/c/vtk-git$ find . | xargs -iX grep -Hs VTK_PYTHON_FULL_THREADSAFE X 
./Utilities/PythonInterpreter/vtkPythonInterpreter.cxx:#ifdef VTK_PYTHON_FULL_THREADSAFE 
./Utilities/Python/vtkPythonConfigure.h.in:#cmakedefine VTK_PYTHON_FULL_THREADSAFE 
./Utilities/Python/vtkPython.h:// Note: behaviour of this class depends on VTK_PYTHON_FULL_THREADSAFE. 
./Utilities/Python/vtkPython.h: // If force is TRUE, lock/unlock even if VTK_PYTHON_FULL_THREADSAFE is not defined. 
./Utilities/Python/vtkPython.h: // If force is FALSE, lock/unlock is only performed if VTK_PYTHON_FULL_THREADSAFE is 
./Utilities/Python/vtkPython.h:#ifdef VTK_PYTHON_FULL_THREADSAFE 
./Utilities/Python/vtkPython.h: // Force is always true with VTK_PYTHON_FULL_THREADSAFE 

/data/c/vtk-git-release$ cmake -D VTK_WRAP_PYTHON=ON ../vtk-git 
... 
/data/c/vtk-git-release$ grep VTK_PYTHON_FULL_THREADSAFE CMakeCache.txt 
<empty> 

/data/c/vtk-git-release$ grep THREAD CMakeCache.txt 
CMAKE_THREAD_LIBS:STRING=-lpthread 
LIBPROJ_USE_THREAD:BOOL=ON 
VTK_MAX_THREADS:STRING=64 
CMAKE_HAVE_PTHREADS_CREATE:INTERNAL= 
CMAKE_HAVE_PTHREAD_CREATE:INTERNAL=1 
CMAKE_HAVE_PTHREAD_H:INTERNAL=1 
//ADVANCED property for variable: CMAKE_THREAD_LIBS 
CMAKE_THREAD_LIBS-ADVANCED:INTERNAL=1 
H5_HAVE_PTHREAD_H:INTERNAL=1 
H5_SYSTEM_SCOPE_THREADS:INTERNAL=1 
HAVE_PTHREAD_H:INTERNAL=1 
//Have symbol PTHREAD_MUTEX_RECURSIVE 
HAVE_PTHREAD_MUTEX_RECURSIVE_DEFN:INTERNAL= 
HDF5_ENABLE_THREADSAFE:INTERNAL=OFF 
//ADVANCED property for variable: LIBPROJ_USE_THREAD 
LIBPROJ_USE_THREAD-ADVANCED:INTERNAL=1 
SYSTEM_SCOPE_THREADS:INTERNAL=TRUE 
//ADVANCED property for variable: VTK_MAX_THREADS 
VTK_MAX_THREADS-ADVANCED:INTERNAL=1 

Paul 


From: "Mathieu Westphal" < mathieu.westphal at kitware.com > 
To: "Paul Melis" < paul.melis at surfsara.nl > 
Cc: "vtk" < vtkusers at vtk.org > 
Sent: Thursday, 8 September, 2016 13:51:18 
Subject: Re: [vtkusers] VTK 7.0.0 + Python: GIL released? 

Hi 

All python calls are protected by gil if the VTK_PYTHON_FULL_THREADSAFE option is on. 
The mechanism take place in vtkPython.h, where the vtkPythonScopeGilEnsurer is configured by the cmake option. 
the vtkPythonScopeGilEnsurer is then ( suposed to be ) used before every pthon call. 

Regards, 

Mathieu Westphal 

On Thu, Sep 8, 2016 at 1:23 PM, Paul Melis < paul.melis at surfsara.nl > wrote: 

BQ_BEGIN
Hi all, 

What's the status of threading in VTK 7.0.0 under Python w.r.t the GIL, 
specifically for classes derived from vtkAlgorithm? 
I'm attempting to offload a computation (basically vtkStreamTracer.Update()) 
to a Python threading.Thread, but strongly get the impression that 
the GIL is not released by Update(). 

The release notes ( https://blog.kitware.com/vtk-7-0-0/ ) seem to suggest 
there was some recent development in this area: 

* multi-threaded Python codes facilitated by VTK's new and optional 
protocol for using the Global Interpreter Lock 

But clicking the link appears to indicate this is for VTK used inside 
Paraview only. I see the mentioned cmake variable VTK_PYTHON_FULL_THREADSAFE 
referenced by files under Utilities/Python, but the cmake files themselves 
don't contain a reference, nor something like an option. Would hacking 
the define in somewhere cause Update() to be protected by the GIL? 
Commit https://github.com/Kitware/VTK/commit/8eb50dd23fd762d42fe1a6785bbaa704f2ed76c3 
lists some classes and their possible protection by vtkPythonScopeGilEnsurer, 
but what specifically is covered isn't entirely clear to me. 

Thanks, 
Paul 

-- 

Paul Melis 
| Visualization group leader & developer | SURFsara | 
| Science Park 140 | 1098 XG Amsterdam | 
| T 020 800 1312 | paul.melis at surfsara.nl | www.surfsara.nl | 
_______________________________________________ 
Powered by www.kitware.com 

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html 

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ 

Search the list archives at: http://markmail.org/search/?q=vtkusers 

Follow this link to subscribe/unsubscribe: 
http://public.kitware.com/mailman/listinfo/vtkusers 





BQ_END


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtkusers/attachments/20160908/344b04ba/attachment.html>