[Paraview] Automating contour isosurface values in parallel Catalyst Python script

Fri Feb 17 04:50:04 EST 2017

Hi Andy and Joachim,

I adapted Andy's suggestions, and my script is now both slightly simpler (no mpi4py or numpy dependencies) and works on both machines.

PS. I cc'd the ParaView mailing list, since I started this thread on it, and forgot "reply to all" in one of my own responses...

Thanks again for the help, and best regards,

  Yvan

----- Mail original -----
De: "Andy Bauer" <andy.bauer at kitware.com>
À: "Yvan Fournier" <yvan.fournier at free.fr>, "Joachim Pouderoux" <joachim.pouderoux at kitware.com>
Envoyé: Jeudi 16 Février 2017 16:15:03
Objet: Re: [Paraview] Automating contour isosurface values in parallel Catalyst Python script

Hi Yvan, 

I inlined some answers below... 

Best, 
Andy 

ps. I cc'ed Joachim since it looks like you meant to cc him but did not. 

On Thu, Feb 16, 2017 at 6:25 AM, < yvan.fournier at free.fr > wrote: 

Hi Andy and Joachim, 

Thanks for your answers. I suspected I would need a solution of this sort. 

I prefer computing the min and max myself rather than adapting a Catalyst edition, as I often use full/default ParaView with OSMesa instead of Catalyst Editions (I only got a "standard" base + Python + extensions... edition working very recently; with older versions of the editions, some filter was always missing and I could not use Python scripts generated by the Coprocessing Generator plugin). 

I generally just use the full PV version also. It is only when I need to be very light on memory that I switch to the Catalyst editions. Others have noted that the Catalyst editions are easier to build on HPC systems though and prefer that route. 

I made good progress, and now have a solution which works with a 1 or 2-week-old build from the master branch, using numpy, but does not work on a build with ParaView 5.1.2 on an older machine due to missing features: 

- "from paraview.vtk.numpy_interface.algorithms import *" fails, complaining it does not find numpy and tells me to check it was installed properly (it is installed, though might have been installed after the ParaView build) 

I do not know about this issue as I have not seen it before and was not involved with the integration. Maybe Joachim knows. Otherwise I would suggest going back to the PV mailing list. 

- using: 
controller = coprocessing.vtkCompositeMultiProcessController.GetGlobalController() 
if controller and controller.IsA("vtkMPIController") and controller.GetNumberOfProcesses() > 1: 
from mpi4py import MPI 
comm = vtkMPI4PyCommunicator.ConvertToPython(controller.GetCommunicator()) 

In cases like this I just use: 

import vtk 

controller = vtk.vtkMultiProcessController.GetGlobalController() 

c.AllReduce(<in data>, <out data>, <length>, vtk.vtkCommunicator.MAX_OP) 

to get the max values. vtkCommunicator has the named constants for operations. The "global" communicator will be the one that Catalyst uses which may be a subcommunicator of MPI_COMM_WORLD. I'm not sure if mpi4py will be over the Catalyst MPI processes or all of them. Also, you may want to try "from paraview import vtk" instead of just "import vtk". Less VTK Python stuff should be imported then but it may be missing something you want. I don't use vtkMPI4PYCommunicator so I cannot say how well that should work for your needs. 

I have a "NameError: global name 'vtkMPI4PyCommunicator' is not defined" message 

In my earlier tests, I tried simply using the AllReduce() method from the controller, with vtkArrays, but I was stuck on finding the correct value for the last argument, which seems to be an integer describing the collective reduction operation (I need min and max). But I was unable to find any examples for this in Python, (whether Googling for examples or printing dict() and (help() of various Python objects). 
The documentation only specifies some methods may not be available in Python... 

Are there named constants describing those operations in the Python wrappers of vtkMPIController ? 
I assume I could find the integer values in the C++ documentation/sources, but I would rather have named constants for readability and in case of future changes. 
I guess using the vtkMPIController would slightly simplify the code and reduce dependancy to some external Python modules. 

Thanks for the help, and best regards, 

Yvan 

----- Mail original ----- 
De: "Andy Bauer" < andy.bauer at kitware.com > 
À: "Joachim Pouderoux" < joachim.pouderoux at kitware.com > 
Cc: "Yvan Fournier" < yvan.fournier at free.fr >, "ParaView" < paraview at paraview.org > 
Envoyé: Mardi 14 Février 2017 21:27:06 
Objet: Re: [Paraview] Automating contour isosurface values in parallel Catalyst Python script 

Hi Yvan, 

Another option is just to compute it yourself. The reason that the global min and max aren't know in Catalyst is that it requires communication but many times the value isn't needed so in order to make Catalyst more efficient it doesn't compute that value for you automatically. In the Python scripts you can import mpi4py or import vtk and use the vtkMultiProcessController to do the global operation. Note that for my way you'll need to call UpdatePipeline(datadescription.GetTime()) before getting the output and globally determining the value. Then you can set the isosurfaces values array with that information. 

If you try this out and have issues, please let us know and we can go from there. 

Cheers, 
Andy 

On Tue, Feb 14, 2017 at 3:02 PM, Joachim Pouderoux < joachim.pouderoux at kitware.com > wrote: 

Hi Yvan, 

Regarding MinMax: this filter is indeed not exposed by any standard Catalyst edition but it is 
easy to add. See for instance in ParaView/Catalyst/Editions/Extras/manifest.json how the 
PVExtractSelection filter is exposed (line 50 and 247). 
Just do the same with the MinMax filter (see proxy's declaration at filters.xml:~11725). 

Best, 
Joachim 

Joachim Pouderoux , PhD 

Technical Expert - Scientific Computing Team 
Kitware SAS 

2017-02-13 17:31 GMT-04:00 Yvan Fournier < yvan.fournier at free.fr > : 

Hello, 

I'm in the process of migrating some VTK scripts in the Code_Saturne (code- 
saturne.org ) test suite to Catalyst Python scripts, and am having some 

difficulty automating some cases using contour plots: 

For time-dependent data, I do not know the value ranges in advance, and would 
like to query them so as to build an Isosurface values array spanning that 
range. 

I can manage to access the point and cell data arrays and ther ranges, but 
printing the ranges in parallel, it seems I have local values only, while I need 
a global range. 

I searched for examples in the documentation, on the Wiki, ..., and found an 
example for MinMax, but don't seem to ba able to access the correct type of 
Proxy under Catalyst (the default servermanager is not connected in the usual 
way. 

For scalar bar color look-up tables, I can manage with 
"coprocessor.WriteImages(datadescription, rescale_lookuptable=True)" in the 
coprocessing script, but for a more general case such as thar needed for contour 
plots, I've been running around in circles in the last hours trying to find and 
adapt a relevant example. 

Does anyone have a suggestion ? Thanks in advance, 

Yvan Fournier 
_______________________________________________ 
Powered by www.kitware.com 

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html 

Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView 

Search the list archives at: http://markmail.org/search/?q=ParaView 

Follow this link to subscribe/unsubscribe: 
http://public.kitware.com/mailman/listinfo/paraview 

_______________________________________________ 
Powered by www.kitware.com 

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html 

Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView 

Search the list archives at: http://markmail.org/search/?q=ParaView 

Follow this link to subscribe/unsubscribe: 
http://public.kitware.com/mailman/listinfo/paraview 