[Paraview] Newbie question on Python Programmable Filters

Andy Bauer andy.bauer at kitware.com
Thu Sep 6 21:52:03 EDT 2012 

It's even simpler than setting CCFLAGS to turn debug information on. Just
set CMAKE_BUILD_TYPE to "Debug". It may already be set to that.

If you're creating your VTK files from Python code, you may want to look at
PyEVTK (https://bitbucket.org/pauloh/pyevtk). I haven't tried the
unstructured grid/vtu writer from that but have tried the image data writer
from it.

Andy

On Thu, Sep 6, 2012 at 1:19 PM, Frank Horowitz
<frank.horowitz at cornell.edu>wrote:

> Thanks for the replies!
>
> At the moment, I need to finish a poster for a presentation on Saturday,
> but promise to get to this Sunday or Monday. (I'll need to re-build a
> debugging version of paraview. Any hints on how to accomplish this within
> ccmake/cmake will be gratefully received, since that system is new to me
> too. Is it as simple as setting a CCFLAGS environment variable before
> invoking make -- or something similar??)
>
> I also found that the 200**3 wavelet worked fine on my
> paraview-compiled-from-source Linux installation. Let's ignore the problem
> I found on the OSX box with the binary installer for the moment. (One
> problem at a time!)
>
> Because of that, I'm now suspicious of the upstream code that writes .vtu
> files.
>
> Also, responding here to your later comment from your next email:
>
> > I am going to disagree with Andy about this being a better solution. I
> > don't even think that it addresses what Frank is trying to do. But beyond
> > that Array Calculator is cumbersome and slow compared to the Python
> > Calculator or Python Programmable Filter.
> >
> > As far as I understand, Frank is not trying to combine components from
> > different files. He is trying to combine components written as different
> > arrays in the same file hence the need for sqrt(x*x + y*y + z*z). I think
> > that Andy is confused by inputs[0] - that's simple the way to access the
> > input (even if there is only one) of the Python Programmable Filter.
>
> Actually, there is one file per component per timeslice.
>
> I've had a quick look at the file format written by my upstream simulator.
> It's very simple.  Maybe I should just write the .vtu files from within my
> own script that reads the upstream meshes to avoid any other potential
> problems with the upstream codes.
>
> Does paraview.simple have an easy way to construct a field vector at the
> location of arbitrary points? If so, I think the Python script to do *that*
> will be faster than me trying to chase all of this down…
>
> Thanks again for the advice!
>
> Cheers,
>         Frank Horowitz
>
>
>
> On 06/09/2012, at 1:00 PM, Berk Geveci wrote:
>
> > I just ran this with a 500^3 wavelet on my machine without any problems.
> > The whole thing (including a slice filter) takes 20-30 seconds but the
> > programmable filter is actually pretty much instantaneous (<0.1 seconds)
> > using a debug build from Git master. Is there any way you can attach to
> the
> > stuck ParaView with a debugger and look at the stack trace?
> >
> > On Wed, Sep 5, 2012 at 9:13 PM, Frank Horowitz
> > <frank.horowitz at cornell.edu>wrote:
> >
> >> Still more checking things out. A problem (THE problem???) can be
> >> demonstrated with your code snippet from a few messages back if you
> >> construct the Wavelet like so rather than using the default constructor:
> >>
> >> Wavelet(WholeExtent=(-100,100,-100,100,-100,100))
> >>
> >> N.B. this is on a different machine, with a stock Mac OSX  binary
> version
> >> 3.14.1 installed.
> >>
> >>
> >> On 05/09/2012, at 5:49 PM, Frank Horowitz wrote:
> >>
> >>> OK, more tracking the problem:
> >>>
> >>> The assumption underlying my "close up" description below is not
> >> correct.  The problem appears to be the actual _size_ of the dataset I'm
> >> feeding in. Even a single time step fed in to the filter (with some
> 866,000
> >> points) hangs at the x*x computation, even though that appeared to work
> >> perfectly fine with your 3D Wavelet as a source (only some 9,000
> points).
> >>>
> >>> I'm guessing a memory leak somewhere?  When was the last time paraview
> >> was run through valgrind?
> >>>
> >>> Cheers (and sorry about the red herring earlier),
> >>>      Frank
> >>>
> >>>
> >>>
> >>> On 05/09/2012, at 4:42 PM, Frank Horowitz wrote:
> >>>
> >>>> OK, the 30,000 foot view:
> >>>>
> >>>> I have a time series of about 20 slices (currently) of values of each
> >> component (3 components total) of a vector displacement 3d spatial field
> >> from an elastic wave propagation simulation. At the moment, I'm simply
> >> trying to visualise the vector length to see the propagating phases of
> the
> >> wavefield. Because my upstream codes and scripts only write individual
> >> components of the vector field, I need to combine the components and
> then
> >> calculate the vector lengths *somewhere*.  The programmable filter
> seemed
> >> like a good place to try that, since once I am past the learning curve
> for
> >> it, I'll have the full flexibilty of numpy for doing almost anything
> else I
> >> want with these and similar data from my upstream simulation codes.
> >>>>
> >>>> Now the 10,000 foot view:
> >>>> I've been trying to simply calculate the sqrt(x*x + y*y + z*z) scalar
> >> field (where x, y, and z are the respective components as a function of
> >> position)  and let that flow on down the pipeline as a new spatial
> field to
> >> be visualised. I assumed (correctly if I'm interpreting your answer
> >> correctly) that if I built the filter to deal with the spatial aspects
> of
> >> the calculation, the time series would be taken care of by paraview
> >> iterating over all files pointed to by inputs[0] (a set of files
> containing
> >> timesteps of the x components), inputs[1] (ditto for the y components),
> and
> >> inputs[2] (ditto for the z components).  The set of *.vtu files for each
> >> component have filenames specifying component and time step (e.g.: x001,
> >> x002, x003, etc., y001, y002, y003, etc., and z001, z002, z003 etc.).
> >>>>
> >>>> Now the close-up:
> >>>> As far as I can tell, after compiling from source from the git
> >> "release" branch (on an up-to-date Linux Mint distro -- which tracks
> Ubuntu
> >> Precise), my approach will work for a set of 3 single timestep arrays
> fed
> >> into the "ports" (or whatever paraview's terminology is for the inputs
> to a
> >> filter in the pipeline). In other words, a single timeslice
> >> (x001,y001,z001) appears on inputs[0:3] (numpy slice notation) and is
> >> processed correctly. (I'll go away and verify that shortly.)  I *think*
> the
> >> "hangs/race-conditions/whatever" start when there are multiple timesteps
> >> presented to each input port.  I was unsure whether the 'inputs[0],
> >> inputs[1], inputs[2]' syntax needed to be extended to deal with time
> series
> >> as above, and could not find documentation on the inputs[] array after
> >> extensive searching -- just a few snippets of example code that were all
> >> consistent with paraview dealing with the time series on its own.
> >>>>
> >>>> I'm inexperienced enough with paraview to not know if I have a
> >> conceptual error with the way I'm approaching this problem, or if I have
> >> simply stumbled over a bug. That's why I've posted to the mailing list!
> >>>>
> >>>> As always, your (and the rest of the list's) help is greatly
> >> appreciated!
> >>>>
> >>>> Cheers,
> >>>>     Frank
> >>>>
> >>>>
> >>>> On 05/09/2012, at 4:01 PM, Andy Bauer wrote:
> >>>>
> >>>>> I'm not sure I completely understand about your inputs[0]. As for
> >> iterating
> >>>>> over time with the python programmable filter, inside the filter it
> >> will
> >>>>> only have access to values from a single time step at each invocation
> >> of
> >>>>> RequestData(). You can think of the order of operations if you have 5
> >> time
> >>>>> steps as:
> >>>>> 1) iterate over each of the 5 time steps (i.e. load a file and update
> >> the
> >>>>> pipeline with that data set)
> >>>>> 2) update the python programmable filter output for the current data
> >> set
> >>>>> resulting from the loaded file
> >>>>>
> >>>>> If you want time statistics for your variables I think you probably
> >> want to
> >>>>> use the calculator filter to square the value at each point and then
> >> use
> >>>>> the temporal statistics filter to get an average of those values.
> After
> >>>>> that you can just multiply the results by the number of time steps.
> >>>>>
> >>>>> If I'm misunderstanding what you're trying to do, I'd suggest giving
> a
> >>>>> description of what you're trying to do and maybe there's a simpler
> >> way of
> >>>>> doing it.
> >>>>>
> >>>>> Andy
> >>>>>
> >>>>> On Wed, Sep 5, 2012 at 2:56 PM, Frank Horowitz
> >>>>> <frank.horowitz at cornell.edu>wrote:
> >>>>>
> >>>>>> OK Andy,
> >>>>>>
> >>>>>> Thanks for that!
> >>>>>>
> >>>>>> I've compiled and installed Paraview from git's release branch, and
> (a
> >>>>>> slight modification) of your code runs correctly. I think that means
> >> that
> >>>>>> my paraview/python integration is semi-sane.
> >>>>>>
> >>>>>> However, my problem still occurs. Recall that I have a time series
> of
> >> .vtu
> >>>>>> files (indexed by integers built-in to their filenames) being fed
> into
> >>>>>> inputs[0] (at least as I understand it).
> >>>>>>
> >>>>>> In my original code snippet, I assumed that the "m = x*x" statement
> >> would
> >>>>>> iterate not only over the point values of the spatial array, but
> also
> >> over
> >>>>>> all timeslices.  Is that a correct assumption? Is that the source of
> >> my
> >>>>>> problems???
> >>>>>>
> >>>>>> Thanks again for your help,
> >>>>>> Frank Horowitz
> >>>>>>
> >>>>>> ------------------------------
> >>>>>> *From:* Andy Bauer [andy.bauer at kitware.com]
> >>>>>> *Sent:* Tuesday, September 04, 2012 7:57 PM
> >>>>>> *To:* Frank Horowitz
> >>>>>> *Cc:* paraview at paraview.org
> >>>>>> *Subject:* Re: [Paraview] Newbie question on Python Programmable
> >> Filters
> >>>>>>
> >>>>>> Hi Frank,
> >>>>>>
> >>>>>> I didn't have a problem with the following on my windows machine
> >> using the
> >>>>>> ParaView 3.14.1 installer.
> >>>>>> =======
> >>>>>> try: paraview.simple
> >>>>>>
> >>>>>> except: from paraview.simple import *
> >>>>>>
> >>>>>> paraview.simple._DisableFirstRenderCameraReset()
> >>>>>>
> >>>>>> Wavelet1 = Wavelet()
> >>>>>>
> >>>>>> RenderView1 = GetRenderView()
> >>>>>>
> >>>>>> ProgrammableFilter1 = ProgrammableFilter()
> >>>>>>
> >>>>>> ProgrammableFilter1.PythonPath = ''
> >>>>>>
> >>>>>> ProgrammableFilter1.RequestInformationScript = ''
> >>>>>>
> >>>>>> ProgrammableFilter1.Script = 'x =
> >> inputs[0].PointData[\'RTData\']\nprint
> >>>>>> "x shape =",x.shape\n\nm = x*x # I _thought_ this should run at
> numpy
> >>>>>> speeds?\nprint "m shape=",m.shape\n\n# more debugging code to finish
> >>>>>> computation of the norm
> >> omitted\n\noutput.PointData.append(m,"Displacement
> >>>>>> Norm")'
> >>>>>>
> >>>>>> Show()
> >>>>>>
> >>>>>> Render()
> >>>>>>
> >>>>>> =============
> >>>>>>
> >>>>>> Does this work for you?  If it doesn't, I wonder if it's an issue
> with
> >>>>>> having multiple pythons and numpy or system environments causing
> >> problems.
> >>>>>> What version of paraview are you using and how did you get it
> >>>>>> built/installed on your machine?
> >>>>>>
> >>>>>> Andy
> >>>>>>
> >>>>>> On Tue, Sep 4, 2012 at 6:04 PM, Frank Horowitz <
> >> frank.horowitz at cornell.edu
> >>>>>>> wrote:
> >>>>>>
> >>>>>>> Hal Canary wrote on Tue Sep 4 15:22:01 EDT 2012:
> >>>>>>>> On 09/04/2012 02:20 PM, Frank Horowitz wrote:
> >>>>>>>>>
> >>>>>>>> x = inputs[0].PointData['Scalars_']
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> I thought one needs to convert a vtkarray to a numpy array with
> >>>>>>>>
> >>>>>>>>   x = numpy.array(inputs[0].PointData['Scalars_'])
> >>>>>>>>
> >>>>>>>> before doing anything with it.
> >>>>>>>
> >>>>>>>
> >>>>>>> Computations after that style of construction hang too.  To my eye,
> >> the
> >>>>>>> results of such an expression should be 100% standard numpy,
> obeying
> >> 100%
> >>>>>>> standard numpy semantics.
> >>>>>>>
> >>>>>>> It appears that there is a bug in numpy integration to the Python
> >>>>>>> Programmable Filter.
> >>>>>>>
> >>>>>>> I'll re-compile from source and report back on the results.  Is the
> >> bug
> >>>>>>> tracking system evident from the homepage?
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>>     Frank Horowitz
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> Powered by www.kitware.com
> >>>>>>>
> >>>>>>> Visit other Kitware open-source projects at
> >>>>>>> http://www.kitware.com/opensource/opensource.html
> >>>>>>>
> >>>>>>> Please keep messages on-topic and check the ParaView Wiki at:
> >>>>>>> http://paraview.org/Wiki/ParaView
> >>>>>>>
> >>>>>>> Follow this link to subscribe/unsubscribe:
> >>>>>>> http://www.paraview.org/mailman/listinfo/paraview
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> >> _______________________________________________
> >> Powered by www.kitware.com
> >>
> >> Visit other Kitware open-source projects at
> >> http://www.kitware.com/opensource/opensource.html
> >>
> >> Please keep messages on-topic and check the ParaView Wiki at:
> >> http://paraview.org/Wiki/ParaView
> >>
> >> Follow this link to subscribe/unsubscribe:
> >> http://www.paraview.org/mailman/listinfo/paraview
> >>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20120906/bb4c57b8/attachment-0001.htm>

More information about the ParaView mailing list