[Paraview] Newbie question on Python Programmable Filters

[Andy Bauer]

Hmm, I tried with the WholeExtent of (-100, 100, -100, 100, -100, 100) on
my machine and still didn't have problems. How much memory is on your
system? I'd hate to think that ParaView is that poor with memory
management. If I remember correctly, we used to have nightly dashboards
running valgrind. I know that Bill Lorensen's project last fall was going
through with valgrind to get rid of a bunch of memory issues.

Well, the good news is that I think I have a better solution for you
anyways. I get the feeling from your original script that you have a
separate file for each time step and each component of your vector field.
If all components of your vector field are in the same file you can ignore
the first part of the steps below. Here's the solution:

1) load all of the files for the first time step (e.g. load the
<filename>..<file extention>
2) use control and select all the file readers
3) use the append attributes filter to combine the fields from each input
data set into a single output dataset (it assumes that the input data sets
all have the same number of points and cells and the desired fields all
have different names -- use the calculator filter to change the field names
if needed)
<if all of the fields are in the same file, start here after loading the
file>
4) create a calculator filter
4a) compute the magnitude by inputting in "mag(<x component field
name>*iHat + <y component field name>*jHat + <z component field
name>*kHat)" for the widget right above the calculator buttons
4b) if you want a specific name for the newly created field, set it in
Result Array Name

Let me know if this accomplishes your goal.

Andy

On Wed, Sep 5, 2012 at 9:13 PM, Frank Horowitz
<frank.horowitz at cornell.edu>wrote:

> Still more checking things out. A problem (THE problem???) can be
> demonstrated with your code snippet from a few messages back if you
> construct the Wavelet like so rather than using the default constructor:
>
> Wavelet(WholeExtent=(-100,100,-100,100,-100,100))
>
> N.B. this is on a different machine, with a stock Mac OSX  binary version
> 3.14.1 installed.
>
>
> On 05/09/2012, at 5:49 PM, Frank Horowitz wrote:
>
> > OK, more tracking the problem:
> >
> > The assumption underlying my "close up" description below is not
> correct.  The problem appears to be the actual _size_ of the dataset I'm
> feeding in. Even a single time step fed in to the filter (with some 866,000
> points) hangs at the x*x computation, even though that appeared to work
> perfectly fine with your 3D Wavelet as a source (only some 9,000 points).
> >
> > I'm guessing a memory leak somewhere?  When was the last time paraview
> was run through valgrind?
> >
> > Cheers (and sorry about the red herring earlier),
> >       Frank
> >
> >
> >
> > On 05/09/2012, at 4:42 PM, Frank Horowitz wrote:
> >
> >> OK, the 30,000 foot view:
> >>
> >> I have a time series of about 20 slices (currently) of values of each
> component (3 components total) of a vector displacement 3d spatial field
> from an elastic wave propagation simulation. At the moment, I'm simply
> trying to visualise the vector length to see the propagating phases of the
> wavefield. Because my upstream codes and scripts only write individual
> components of the vector field, I need to combine the components and then
> calculate the vector lengths *somewhere*.  The programmable filter seemed
> like a good place to try that, since once I am past the learning curve for
> it, I'll have the full flexibilty of numpy for doing almost anything else I
> want with these and similar data from my upstream simulation codes.
> >>
> >> Now the 10,000 foot view:
> >> I've been trying to simply calculate the sqrt(x*x + y*y + z*z) scalar
> field (where x, y, and z are the respective components as a function of
> position)  and let that flow on down the pipeline as a new spatial field to
> be visualised. I assumed (correctly if I'm interpreting your answer
> correctly) that if I built the filter to deal with the spatial aspects of
> the calculation, the time series would be taken care of by paraview
> iterating over all files pointed to by inputs[0] (a set of files containing
> timesteps of the x components), inputs[1] (ditto for the y components), and
> inputs[2] (ditto for the z components).  The set of *.vtu files for each
> component have filenames specifying component and time step (e.g.: x001,
> x002, x003, etc., y001, y002, y003, etc., and z001, z002, z003 etc.).
> >>
> >> Now the close-up:
> >> As far as I can tell, after compiling from source from the git
> "release" branch (on an up-to-date Linux Mint distro -- which tracks Ubuntu
> Precise), my approach will work for a set of 3 single timestep arrays fed
> into the "ports" (or whatever paraview's terminology is for the inputs to a
> filter in the pipeline). In other words, a single timeslice
> (x001,y001,z001) appears on inputs[0:3] (numpy slice notation) and is
> processed correctly. (I'll go away and verify that shortly.)  I *think* the
> "hangs/race-conditions/whatever" start when there are multiple timesteps
> presented to each input port.  I was unsure whether the 'inputs[0],
> inputs[1], inputs[2]' syntax needed to be extended to deal with time series
> as above, and could not find documentation on the inputs[] array after
> extensive searching -- just a few snippets of example code that were all
> consistent with paraview dealing with the time series on its own.
> >>
> >> I'm inexperienced enough with paraview to not know if I have a
> conceptual error with the way I'm approaching this problem, or if I have
> simply stumbled over a bug. That's why I've posted to the mailing list!
> >>
> >> As always, your (and the rest of the list's) help is greatly
> appreciated!
> >>
> >> Cheers,
> >>      Frank
> >>
> >>
> >> On 05/09/2012, at 4:01 PM, Andy Bauer wrote:
> >>
> >>> I'm not sure I completely understand about your inputs[0]. As for
> iterating
> >>> over time with the python programmable filter, inside the filter it
> will
> >>> only have access to values from a single time step at each invocation
> of
> >>> RequestData(). You can think of the order of operations if you have 5
> time
> >>> steps as:
> >>> 1) iterate over each of the 5 time steps (i.e. load a file and update
> the
> >>> pipeline with that data set)
> >>> 2) update the python programmable filter output for the current data
> set
> >>> resulting from the loaded file
> >>>
> >>> If you want time statistics for your variables I think you probably
> want to
> >>> use the calculator filter to square the value at each point and then
> use
> >>> the temporal statistics filter to get an average of those values. After
> >>> that you can just multiply the results by the number of time steps.
> >>>
> >>> If I'm misunderstanding what you're trying to do, I'd suggest giving a
> >>> description of what you're trying to do and maybe there's a simpler
> way of
> >>> doing it.
> >>>
> >>> Andy
> >>>
> >>> On Wed, Sep 5, 2012 at 2:56 PM, Frank Horowitz
> >>> <frank.horowitz at cornell.edu>wrote:
> >>>
> >>>> OK Andy,
> >>>>
> >>>> Thanks for that!
> >>>>
> >>>> I've compiled and installed Paraview from git's release branch, and (a
> >>>> slight modification) of your code runs correctly. I think that means
> that
> >>>> my paraview/python integration is semi-sane.
> >>>>
> >>>> However, my problem still occurs. Recall that I have a time series of
> .vtu
> >>>> files (indexed by integers built-in to their filenames) being fed into
> >>>> inputs[0] (at least as I understand it).
> >>>>
> >>>> In my original code snippet, I assumed that the "m = x*x" statement
> would
> >>>> iterate not only over the point values of the spatial array, but also
> over
> >>>> all timeslices.  Is that a correct assumption? Is that the source of
> my
> >>>> problems???
> >>>>
> >>>> Thanks again for your help,
> >>>>  Frank Horowitz
> >>>>
> >>>> ------------------------------
> >>>> *From:* Andy Bauer [andy.bauer at kitware.com]
> >>>> *Sent:* Tuesday, September 04, 2012 7:57 PM
> >>>> *To:* Frank Horowitz
> >>>> *Cc:* paraview at paraview.org
> >>>> *Subject:* Re: [Paraview] Newbie question on Python Programmable
> Filters
> >>>>
> >>>> Hi Frank,
> >>>>
> >>>> I didn't have a problem with the following on my windows machine
> using the
> >>>> ParaView 3.14.1 installer.
> >>>> =======
> >>>> try: paraview.simple
> >>>>
> >>>> except: from paraview.simple import *
> >>>>
> >>>> paraview.simple._DisableFirstRenderCameraReset()
> >>>>
> >>>> Wavelet1 = Wavelet()
> >>>>
> >>>> RenderView1 = GetRenderView()
> >>>>
> >>>> ProgrammableFilter1 = ProgrammableFilter()
> >>>>
> >>>> ProgrammableFilter1.PythonPath = ''
> >>>>
> >>>> ProgrammableFilter1.RequestInformationScript = ''
> >>>>
> >>>> ProgrammableFilter1.Script = 'x =
> inputs[0].PointData[\'RTData\']\nprint
> >>>> "x shape =",x.shape\n\nm = x*x # I _thought_ this should run at numpy
> >>>> speeds?\nprint "m shape=",m.shape\n\n# more debugging code to finish
> >>>> computation of the norm
> omitted\n\noutput.PointData.append(m,"Displacement
> >>>> Norm")'
> >>>>
> >>>> Show()
> >>>>
> >>>> Render()
> >>>>
> >>>> =============
> >>>>
> >>>> Does this work for you?  If it doesn't, I wonder if it's an issue with
> >>>> having multiple pythons and numpy or system environments causing
> problems.
> >>>> What version of paraview are you using and how did you get it
> >>>> built/installed on your machine?
> >>>>
> >>>> Andy
> >>>>
> >>>> On Tue, Sep 4, 2012 at 6:04 PM, Frank Horowitz <
> frank.horowitz at cornell.edu
> >>>>> wrote:
> >>>>
> >>>>> Hal Canary wrote on Tue Sep 4 15:22:01 EDT 2012:
> >>>>>> On 09/04/2012 02:20 PM, Frank Horowitz wrote:
> >>>>>>>
> >>>>>> x = inputs[0].PointData['Scalars_']
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> I thought one needs to convert a vtkarray to a numpy array with
> >>>>>>
> >>>>>>    x = numpy.array(inputs[0].PointData['Scalars_'])
> >>>>>>
> >>>>>> before doing anything with it.
> >>>>>
> >>>>>
> >>>>> Computations after that style of construction hang too.  To my eye,
> the
> >>>>> results of such an expression should be 100% standard numpy, obeying
> 100%
> >>>>> standard numpy semantics.
> >>>>>
> >>>>> It appears that there is a bug in numpy integration to the Python
> >>>>> Programmable Filter.
> >>>>>
> >>>>> I'll re-compile from source and report back on the results.  Is the
> bug
> >>>>> tracking system evident from the homepage?
> >>>>>
> >>>>> Cheers,
> >>>>>      Frank Horowitz
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Powered by www.kitware.com
> >>>>>
> >>>>> Visit other Kitware open-source projects at
> >>>>> http://www.kitware.com/opensource/opensource.html
> >>>>>
> >>>>> Please keep messages on-topic and check the ParaView Wiki at:
> >>>>> http://paraview.org/Wiki/ParaView
> >>>>>
> >>>>> Follow this link to subscribe/unsubscribe:
> >>>>> http://www.paraview.org/mailman/listinfo/paraview
> >>>>>
> >>>>
> >>>>
> >>
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20120906/934f58bb/attachment-0001.htm>