[Paraview] Newbie question on Python Programmable Filters

Thu Sep 6 13:00:54 EDT 2012

I just ran this with a 500^3 wavelet on my machine without any problems.
The whole thing (including a slice filter) takes 20-30 seconds but the
programmable filter is actually pretty much instantaneous (<0.1 seconds)
using a debug build from Git master. Is there any way you can attach to the
stuck ParaView with a debugger and look at the stack trace?

On Wed, Sep 5, 2012 at 9:13 PM, Frank Horowitz
<frank.horowitz at cornell.edu>wrote:

> Still more checking things out. A problem (THE problem???) can be
> demonstrated with your code snippet from a few messages back if you
> construct the Wavelet like so rather than using the default constructor:
>
> Wavelet(WholeExtent=(-100,100,-100,100,-100,100))
>
> N.B. this is on a different machine, with a stock Mac OSX  binary version
> 3.14.1 installed.
>
>
> On 05/09/2012, at 5:49 PM, Frank Horowitz wrote:
>
> > OK, more tracking the problem:
> >
> > The assumption underlying my "close up" description below is not
> correct.  The problem appears to be the actual _size_ of the dataset I'm
> feeding in. Even a single time step fed in to the filter (with some 866,000
> points) hangs at the x*x computation, even though that appeared to work
> perfectly fine with your 3D Wavelet as a source (only some 9,000 points).
> >
> > I'm guessing a memory leak somewhere?  When was the last time paraview
> was run through valgrind?
> >
> > Cheers (and sorry about the red herring earlier),
> >       Frank
> >
> >
> >
> > On 05/09/2012, at 4:42 PM, Frank Horowitz wrote:
> >
> >> OK, the 30,000 foot view:
> >>
> >> I have a time series of about 20 slices (currently) of values of each
> component (3 components total) of a vector displacement 3d spatial field
> from an elastic wave propagation simulation. At the moment, I'm simply
> trying to visualise the vector length to see the propagating phases of the
> wavefield. Because my upstream codes and scripts only write individual
> components of the vector field, I need to combine the components and then
> calculate the vector lengths *somewhere*.  The programmable filter seemed
> like a good place to try that, since once I am past the learning curve for
> it, I'll have the full flexibilty of numpy for doing almost anything else I
> want with these and similar data from my upstream simulation codes.
> >>
> >> Now the 10,000 foot view:
> >> I've been trying to simply calculate the sqrt(x*x + y*y + z*z) scalar
> field (where x, y, and z are the respective components as a function of
> position)  and let that flow on down the pipeline as a new spatial field to
> be visualised. I assumed (correctly if I'm interpreting your answer
> correctly) that if I built the filter to deal with the spatial aspects of
> the calculation, the time series would be taken care of by paraview
> iterating over all files pointed to by inputs[0] (a set of files containing
> timesteps of the x components), inputs[1] (ditto for the y components), and
> inputs[2] (ditto for the z components).  The set of *.vtu files for each
> component have filenames specifying component and time step (e.g.: x001,
> x002, x003, etc., y001, y002, y003, etc., and z001, z002, z003 etc.).
> >>
> >> Now the close-up:
> >> As far as I can tell, after compiling from source from the git
> "release" branch (on an up-to-date Linux Mint distro -- which tracks Ubuntu
> Precise), my approach will work for a set of 3 single timestep arrays fed
> into the "ports" (or whatever paraview's terminology is for the inputs to a
> filter in the pipeline). In other words, a single timeslice
> (x001,y001,z001) appears on inputs[0:3] (numpy slice notation) and is
> processed correctly. (I'll go away and verify that shortly.)  I *think* the
> "hangs/race-conditions/whatever" start when there are multiple timesteps
> presented to each input port.  I was unsure whether the 'inputs[0],
> inputs[1], inputs[2]' syntax needed to be extended to deal with time series
> as above, and could not find documentation on the inputs[] array after
> extensive searching -- just a few snippets of example code that were all
> consistent with paraview dealing with the time series on its own.
> >>
> >> I'm inexperienced enough with paraview to not know if I have a
> conceptual error with the way I'm approaching this problem, or if I have
> simply stumbled over a bug. That's why I've posted to the mailing list!
> >>
> >> As always, your (and the rest of the list's) help is greatly
> appreciated!
> >>
> >> Cheers,
> >>      Frank
> >>
> >>
> >> On 05/09/2012, at 4:01 PM, Andy Bauer wrote:
> >>
> >>> I'm not sure I completely understand about your inputs[0]. As for
> iterating
> >>> over time with the python programmable filter, inside the filter it
> will
> >>> only have access to values from a single time step at each invocation
> of
> >>> RequestData(). You can think of the order of operations if you have 5
> time
> >>> steps as:
> >>> 1) iterate over each of the 5 time steps (i.e. load a file and update
> the
> >>> pipeline with that data set)
> >>> 2) update the python programmable filter output for the current data
> set
> >>> resulting from the loaded file
> >>>
> >>> If you want time statistics for your variables I think you probably
> want to
> >>> use the calculator filter to square the value at each point and then
> use
> >>> the temporal statistics filter to get an average of those values. After
> >>> that you can just multiply the results by the number of time steps.
> >>>
> >>> If I'm misunderstanding what you're trying to do, I'd suggest giving a
> >>> description of what you're trying to do and maybe there's a simpler
> way of
> >>> doing it.
> >>>
> >>> Andy
> >>>
> >>> On Wed, Sep 5, 2012 at 2:56 PM, Frank Horowitz
> >>> <frank.horowitz at cornell.edu>wrote:
> >>>
> >>>> OK Andy,
> >>>>
> >>>> Thanks for that!
> >>>>
> >>>> I've compiled and installed Paraview from git's release branch, and (a
> >>>> slight modification) of your code runs correctly. I think that means
> that
> >>>> my paraview/python integration is semi-sane.
> >>>>
> >>>> However, my problem still occurs. Recall that I have a time series of
> .vtu
> >>>> files (indexed by integers built-in to their filenames) being fed into
> >>>> inputs[0] (at least as I understand it).
> >>>>
> >>>> In my original code snippet, I assumed that the "m = x*x" statement
> would
> >>>> iterate not only over the point values of the spatial array, but also
> over
> >>>> all timeslices.  Is that a correct assumption? Is that the source of
> my
> >>>> problems???
> >>>>
> >>>> Thanks again for your help,
> >>>>  Frank Horowitz
> >>>>
> >>>> ------------------------------
> >>>> *From:* Andy Bauer [andy.bauer at kitware.com]
> >>>> *Sent:* Tuesday, September 04, 2012 7:57 PM
> >>>> *To:* Frank Horowitz
> >>>> *Cc:* paraview at paraview.org
> >>>> *Subject:* Re: [Paraview] Newbie question on Python Programmable
> Filters
> >>>>
> >>>> Hi Frank,
> >>>>
> >>>> I didn't have a problem with the following on my windows machine
> using the
> >>>> ParaView 3.14.1 installer.
> >>>> =======
> >>>> try: paraview.simple
> >>>>
> >>>> except: from paraview.simple import *
> >>>>
> >>>> paraview.simple._DisableFirstRenderCameraReset()
> >>>>
> >>>> Wavelet1 = Wavelet()
> >>>>
> >>>> RenderView1 = GetRenderView()
> >>>>
> >>>> ProgrammableFilter1 = ProgrammableFilter()
> >>>>
> >>>> ProgrammableFilter1.PythonPath = ''
> >>>>
> >>>> ProgrammableFilter1.RequestInformationScript = ''
> >>>>
> >>>> ProgrammableFilter1.Script = 'x =
> inputs[0].PointData[\'RTData\']\nprint
> >>>> "x shape =",x.shape\n\nm = x*x # I _thought_ this should run at numpy
> >>>> speeds?\nprint "m shape=",m.shape\n\n# more debugging code to finish
> >>>> computation of the norm
> omitted\n\noutput.PointData.append(m,"Displacement
> >>>> Norm")'
> >>>>
> >>>> Show()
> >>>>
> >>>> Render()
> >>>>
> >>>> =============
> >>>>
> >>>> Does this work for you?  If it doesn't, I wonder if it's an issue with
> >>>> having multiple pythons and numpy or system environments causing
> problems.
> >>>> What version of paraview are you using and how did you get it
> >>>> built/installed on your machine?
> >>>>
> >>>> Andy
> >>>>
> >>>> On Tue, Sep 4, 2012 at 6:04 PM, Frank Horowitz <
> frank.horowitz at cornell.edu
> >>>>> wrote:
> >>>>
> >>>>> Hal Canary wrote on Tue Sep 4 15:22:01 EDT 2012:
> >>>>>> On 09/04/2012 02:20 PM, Frank Horowitz wrote:
> >>>>>>>
> >>>>>> x = inputs[0].PointData['Scalars_']
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> I thought one needs to convert a vtkarray to a numpy array with
> >>>>>>
> >>>>>>    x = numpy.array(inputs[0].PointData['Scalars_'])
> >>>>>>
> >>>>>> before doing anything with it.
> >>>>>
> >>>>>
> >>>>> Computations after that style of construction hang too.  To my eye,
> the
> >>>>> results of such an expression should be 100% standard numpy, obeying
> 100%
> >>>>> standard numpy semantics.
> >>>>>
> >>>>> It appears that there is a bug in numpy integration to the Python
> >>>>> Programmable Filter.
> >>>>>
> >>>>> I'll re-compile from source and report back on the results.  Is the
> bug
> >>>>> tracking system evident from the homepage?
> >>>>>
> >>>>> Cheers,
> >>>>>      Frank Horowitz
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Powered by www.kitware.com
> >>>>>
> >>>>> Visit other Kitware open-source projects at
> >>>>> http://www.kitware.com/opensource/opensource.html
> >>>>>
> >>>>> Please keep messages on-topic and check the ParaView Wiki at:
> >>>>> http://paraview.org/Wiki/ParaView
> >>>>>
> >>>>> Follow this link to subscribe/unsubscribe:
> >>>>> http://www.paraview.org/mailman/listinfo/paraview
> >>>>>
> >>>>
> >>>>
> >>
> >
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the ParaView Wiki at:
> http://paraview.org/Wiki/ParaView
>
> Follow this link to subscribe/unsubscribe:
> http://www.paraview.org/mailman/listinfo/paraview
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20120906/37d00312/attachment-0001.htm>