[Paraview] Newbie question on Python Programmable Filters

Thu Sep 6 12:21:56 EDT 2012

Given the trivial amount of data involved, I can't imagine this being a
memory leak. Maybe if it hung after 10 time steps... Do you have a way of
verifying that your numpy installation works properly? So for example, what
happens if you do this in the programmable filter:

import numpy
a = numpy.arange(1000000)
m = a * a

-berk

On Wed, Sep 5, 2012 at 5:49 PM, Frank Horowitz
<frank.horowitz at cornell.edu>wrote:

> OK, more tracking the problem:
>
> The assumption underlying my "close up" description below is not correct.
>  The problem appears to be the actual _size_ of the dataset I'm feeding in.
> Even a single time step fed in to the filter (with some 866,000 points)
> hangs at the x*x computation, even though that appeared to work perfectly
> fine with your 3D Wavelet as a source (only some 9,000 points).
>
> I'm guessing a memory leak somewhere?  When was the last time paraview was
> run through valgrind?
>
> Cheers (and sorry about the red herring earlier),
>         Frank
>
>
>
> On 05/09/2012, at 4:42 PM, Frank Horowitz wrote:
>
> > OK, the 30,000 foot view:
> >
> > I have a time series of about 20 slices (currently) of values of each
> component (3 components total) of a vector displacement 3d spatial field
> from an elastic wave propagation simulation. At the moment, I'm simply
> trying to visualise the vector length to see the propagating phases of the
> wavefield. Because my upstream codes and scripts only write individual
> components of the vector field, I need to combine the components and then
> calculate the vector lengths *somewhere*.  The programmable filter seemed
> like a good place to try that, since once I am past the learning curve for
> it, I'll have the full flexibilty of numpy for doing almost anything else I
> want with these and similar data from my upstream simulation codes.
> >
> > Now the 10,000 foot view:
> > I've been trying to simply calculate the sqrt(x*x + y*y + z*z) scalar
> field (where x, y, and z are the respective components as a function of
> position)  and let that flow on down the pipeline as a new spatial field to
> be visualised. I assumed (correctly if I'm interpreting your answer
> correctly) that if I built the filter to deal with the spatial aspects of
> the calculation, the time series would be taken care of by paraview
> iterating over all files pointed to by inputs[0] (a set of files containing
> timesteps of the x components), inputs[1] (ditto for the y components), and
> inputs[2] (ditto for the z components).  The set of *.vtu files for each
> component have filenames specifying component and time step (e.g.: x001,
> x002, x003, etc., y001, y002, y003, etc., and z001, z002, z003 etc.).
> >
> > Now the close-up:
> > As far as I can tell, after compiling from source from the git "release"
> branch (on an up-to-date Linux Mint distro -- which tracks Ubuntu Precise),
> my approach will work for a set of 3 single timestep arrays fed into the
> "ports" (or whatever paraview's terminology is for the inputs to a filter
> in the pipeline). In other words, a single timeslice (x001,y001,z001)
> appears on inputs[0:3] (numpy slice notation) and is processed correctly.
> (I'll go away and verify that shortly.)  I *think* the
> "hangs/race-conditions/whatever" start when there are multiple timesteps
> presented to each input port.  I was unsure whether the 'inputs[0],
> inputs[1], inputs[2]' syntax needed to be extended to deal with time series
> as above, and could not find documentation on the inputs[] array after
> extensive searching -- just a few snippets of example code that were all
> consistent with paraview dealing with the time series on its own.
> >
> > I'm inexperienced enough with paraview to not know if I have a
> conceptual error with the way I'm approaching this problem, or if I have
> simply stumbled over a bug. That's why I've posted to the mailing list!
> >
> > As always, your (and the rest of the list's) help is greatly appreciated!
> >
> > Cheers,
> >       Frank
> >
> >
> > On 05/09/2012, at 4:01 PM, Andy Bauer wrote:
> >
> >> I'm not sure I completely understand about your inputs[0]. As for
> iterating
> >> over time with the python programmable filter, inside the filter it will
> >> only have access to values from a single time step at each invocation of
> >> RequestData(). You can think of the order of operations if you have 5
> time
> >> steps as:
> >> 1) iterate over each of the 5 time steps (i.e. load a file and update
> the
> >> pipeline with that data set)
> >> 2) update the python programmable filter output for the current data set
> >> resulting from the loaded file
> >>
> >> If you want time statistics for your variables I think you probably
> want to
> >> use the calculator filter to square the value at each point and then use
> >> the temporal statistics filter to get an average of those values. After
> >> that you can just multiply the results by the number of time steps.
> >>
> >> If I'm misunderstanding what you're trying to do, I'd suggest giving a
> >> description of what you're trying to do and maybe there's a simpler way
> of
> >> doing it.
> >>
> >> Andy
> >>
> >> On Wed, Sep 5, 2012 at 2:56 PM, Frank Horowitz
> >> <frank.horowitz at cornell.edu>wrote:
> >>
> >>> OK Andy,
> >>>
> >>> Thanks for that!
> >>>
> >>> I've compiled and installed Paraview from git's release branch, and (a
> >>> slight modification) of your code runs correctly. I think that means
> that
> >>> my paraview/python integration is semi-sane.
> >>>
> >>> However, my problem still occurs. Recall that I have a time series of
> .vtu
> >>> files (indexed by integers built-in to their filenames) being fed into
> >>> inputs[0] (at least as I understand it).
> >>>
> >>> In my original code snippet, I assumed that the "m = x*x" statement
> would
> >>> iterate not only over the point values of the spatial array, but also
> over
> >>> all timeslices.  Is that a correct assumption? Is that the source of my
> >>> problems???
> >>>
> >>> Thanks again for your help,
> >>>   Frank Horowitz
> >>>
> >>> ------------------------------
> >>> *From:* Andy Bauer [andy.bauer at kitware.com]
> >>> *Sent:* Tuesday, September 04, 2012 7:57 PM
> >>> *To:* Frank Horowitz
> >>> *Cc:* paraview at paraview.org
> >>> *Subject:* Re: [Paraview] Newbie question on Python Programmable
> Filters
> >>>
> >>> Hi Frank,
> >>>
> >>> I didn't have a problem with the following on my windows machine using
> the
> >>> ParaView 3.14.1 installer.
> >>> =======
> >>> try: paraview.simple
> >>>
> >>> except: from paraview.simple import *
> >>>
> >>> paraview.simple._DisableFirstRenderCameraReset()
> >>>
> >>> Wavelet1 = Wavelet()
> >>>
> >>> RenderView1 = GetRenderView()
> >>>
> >>> ProgrammableFilter1 = ProgrammableFilter()
> >>>
> >>> ProgrammableFilter1.PythonPath = ''
> >>>
> >>> ProgrammableFilter1.RequestInformationScript = ''
> >>>
> >>> ProgrammableFilter1.Script = 'x =
> inputs[0].PointData[\'RTData\']\nprint
> >>> "x shape =",x.shape\n\nm = x*x # I _thought_ this should run at numpy
> >>> speeds?\nprint "m shape=",m.shape\n\n# more debugging code to finish
> >>> computation of the norm
> omitted\n\noutput.PointData.append(m,"Displacement
> >>> Norm")'
> >>>
> >>> Show()
> >>>
> >>> Render()
> >>>
> >>> =============
> >>>
> >>> Does this work for you?  If it doesn't, I wonder if it's an issue with
> >>> having multiple pythons and numpy or system environments causing
> problems.
> >>> What version of paraview are you using and how did you get it
> >>> built/installed on your machine?
> >>>
> >>> Andy
> >>>
> >>> On Tue, Sep 4, 2012 at 6:04 PM, Frank Horowitz <
> frank.horowitz at cornell.edu
> >>>> wrote:
> >>>
> >>>> Hal Canary wrote on Tue Sep 4 15:22:01 EDT 2012:
> >>>>> On 09/04/2012 02:20 PM, Frank Horowitz wrote:
> >>>>>>
> >>>>> x = inputs[0].PointData['Scalars_']
> >>>>>
> >>>>>
> >>>>>
> >>>>> I thought one needs to convert a vtkarray to a numpy array with
> >>>>>
> >>>>>     x = numpy.array(inputs[0].PointData['Scalars_'])
> >>>>>
> >>>>> before doing anything with it.
> >>>>
> >>>>
> >>>> Computations after that style of construction hang too.  To my eye,
> the
> >>>> results of such an expression should be 100% standard numpy, obeying
> 100%
> >>>> standard numpy semantics.
> >>>>
> >>>> It appears that there is a bug in numpy integration to the Python
> >>>> Programmable Filter.
> >>>>
> >>>> I'll re-compile from source and report back on the results.  Is the
> bug
> >>>> tracking system evident from the homepage?
> >>>>
> >>>> Cheers,
> >>>>       Frank Horowitz
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> Powered by www.kitware.com
> >>>>
> >>>> Visit other Kitware open-source projects at
> >>>> http://www.kitware.com/opensource/opensource.html
> >>>>
> >>>> Please keep messages on-topic and check the ParaView Wiki at:
> >>>> http://paraview.org/Wiki/ParaView
> >>>>
> >>>> Follow this link to subscribe/unsubscribe:
> >>>> http://www.paraview.org/mailman/listinfo/paraview
> >>>>
> >>>
> >>>
> >
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the ParaView Wiki at:
> http://paraview.org/Wiki/ParaView
>
> Follow this link to subscribe/unsubscribe:
> http://www.paraview.org/mailman/listinfo/paraview
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20120906/cc2d2496/attachment.htm>