[Paraview-developers] Scipy with ParaView is broken (and Py_NewInterpreter discussion)

[Pat Marion]

A third use is the Reset button on the python shell.  The reset button is
also triggered when you disconnect from a server.  The reset button is
implemented by destroying the interpreter and creating a new one.

Pat


On Fri, Aug 3, 2012 at 5:13 PM, Utkarsh Ayachit <utkarsh.ayachit at kitware.com
> wrote:

> Pat,
>
> I am not surprised. I remember running into this ages ago. If I
> remember correctly, here are the two use-cases for Python in ParaView:
>
> * Qt Python testing uses the python interpreter from a separate thread
> * Python shell, Programmable Filter use Python interpreter from master
> thread but they need separate scopes.
>
> So long as these use-cases can be satisfied, it would make sense to
> not rely on Py_NewInter... if possible.
>
> Utkarsh
>
> p.s. flying today, so we can talk more on Monday, if you'd like.
>
> On Fri, Aug 3, 2012 at 2:16 PM, Pat Marion <pat.marion at kitware.com> wrote:
> > I am using numpy 1.6.1 and scipy 0.9.0 via system packages python-numpy
> and
> > python-scipy on Ubuntu 12.04.  In both the paraview python console and
> > programmable filter, importing certain scipy modules, for example,
> > scipy.stats, causes paraview to hang.  This issue is related to the
> python
> > global interpreter lock (GIL), the program hangs on PyGILState_Ensure().
> > Here is the backtrace:
> >
> >
> > #0  sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:86
> > #1  0x00007ffff1e90c28 in PyThread_acquire_lock () from
> > /usr/lib/libpython2.7.so.1.0
> > #2  0x00007ffff1e91b34 in PyEval_RestoreThread () from
> > /usr/lib/libpython2.7.so.1.0
> > #3  0x00007ffff1e58d08 in PyGILState_Ensure () from
> > /usr/lib/libpython2.7.so.1.0
> > #4  0x00007fffca31ffc6 in ?? () from
> > /usr/lib/python2.7/dist-packages/numpy/core/umath.so
> > #5  0x00007fffca320357 in ?? () from
> > /usr/lib/python2.7/dist-packages/numpy/core/umath.so
> > #6  0x00007fffca337633 in ?? () from
> > /usr/lib/python2.7/dist-packages/numpy/core/umath.so
> > #7  0x00007fffca337ac6 in ?? () from
> > /usr/lib/python2.7/dist-packages/numpy/core/umath.so
> > #8  0x00007ffff1e47e83 in PyObject_Call () from
> /usr/lib/libpython2.7.so.1.0
> > ...
> >
> >
> > This could be a bug in scipy/numpy.  The issue does not appear for older
> > versions of numpy/scipy.  But, it is likely caused by the interaction
> with
> > ParaView's usage of Py_NewInterpreter.  The paraview class
> > vtkPVPythonInterpretor uses the Python C-API function Py_NewInterpreter
> in
> > order to create insulated interpreters.  The documentation for
> > Py_NewInterpreter warns users to be very careful mixing it with the GIL.
> > For example, it says:
> >
> > "Note that the PyGILState_*() functions assume there is only one global
> > interpreter (created automatically by Py_Initialize()). Python supports
> the
> > creation of additional interpreters (using Py_NewInterpreter()), but
> mixing
> > multiple interpreters and the PyGILState_*() API is unsupported."
> >
> > and
> >
> > "Also note that combining this functionality with PyGILState_*() APIs is
> > delicate, because these APIs assume a bijection between Python thread
> states
> > and OS-level threads, an assumption broken by the presence of
> > sub-interpreters."
> >
> > (  docs.python.org/c-api/init.html  )
> >
> >
> > Perhaps it is possible to work around this particular hangup by compiling
> > numpy/scipy with threading disabled or some other flag to disable their
> > usage of the GIL.  But, ParaView's usage of Py_NewInterpreter also makes
> it
> > difficult to use other libraries correctly, like PyQt4 or PySide for
> > example.  I that at some in the future we might want to consider removal
> of
> > Py_NewInterpreter from paraview.
> >
> >
> > Pat
> >
> > _______________________________________________
> > Paraview-developers mailing list
> > Paraview-developers at paraview.org
> > http://public.kitware.com/mailman/listinfo/paraview-developers
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview-developers/attachments/20120803/ac49a174/attachment-0001.htm>