[Paraview] Automating contour isosurface values in parallel Catalyst Python script

Fri Feb 17 10:01:04 EST 2017

Glad to hear!

Also, thanks for replying to the list.

On Fri, Feb 17, 2017 at 4:50 AM, <yvan.fournier at free.fr> wrote:

>
> Hi Andy and Joachim,
>
> I adapted Andy's suggestions, and my script is now both slightly simpler
> (no mpi4py or numpy dependencies) and works on both machines.
>
> PS. I cc'd the ParaView mailing list, since I started this thread on it,
> and forgot "reply to all" in one of my own responses...
>
> Thanks again for the help, and best regards,
>
>   Yvan
>
> ----- Mail original -----
> De: "Andy Bauer" <andy.bauer at kitware.com>
> À: "Yvan Fournier" <yvan.fournier at free.fr>, "Joachim Pouderoux" <
> joachim.pouderoux at kitware.com>
> Envoyé: Jeudi 16 Février 2017 16:15:03
> Objet: Re: [Paraview] Automating contour isosurface values in parallel
> Catalyst Python script
>
>
>
> Hi Yvan,
>
> I inlined some answers below...
>
> Best,
> Andy
>
> ps. I cc'ed Joachim since it looks like you meant to cc him but did not.
>
>
>
>
>
>
>
> On Thu, Feb 16, 2017 at 6:25 AM, < yvan.fournier at free.fr > wrote:
>
>
> Hi Andy and Joachim,
>
> Thanks for your answers. I suspected I would need a solution of this sort.
>
> I prefer computing the min and max myself rather than adapting a Catalyst
> edition, as I often use full/default ParaView with OSMesa instead of
> Catalyst Editions (I only got a "standard" base + Python + extensions...
> edition working very recently; with older versions of the editions, some
> filter was always missing and I could not use Python scripts generated by
> the Coprocessing Generator plugin).
>
>
>
> I generally just use the full PV version also. It is only when I need to
> be very light on memory that I switch to the Catalyst editions. Others have
> noted that the Catalyst editions are easier to build on HPC systems though
> and prefer that route.
>
>
>
>
> I made good progress, and now have a solution which works with a 1 or
> 2-week-old build from the master branch, using numpy, but does not work on
> a build with ParaView 5.1.2 on an older machine due to missing features:
>
> - "from paraview.vtk.numpy_interface.algorithms import *" fails,
> complaining it does not find numpy and tells me to check it was installed
> properly (it is installed, though might have been installed after the
> ParaView build)
>
>
>
> I do not know about this issue as I have not seen it before and was not
> involved with the integration. Maybe Joachim knows. Otherwise I would
> suggest going back to the PV mailing list.
>
>
>
>
> - using:
> controller = coprocessing.vtkCompositeMultiProcessContro
> ller.GetGlobalController()
> if controller and controller.IsA("vtkMPIController") and controller.GetNumberOfProcesses()
> > 1:
> from mpi4py import MPI
> comm = vtkMPI4PyCommunicator.ConvertToPython(controller.GetCommunicator())
>
>
>
>
> In cases like this I just use:
>
> import vtk
>
> controller = vtk.vtkMultiProcessController.GetGlobalController()
>
> c.AllReduce(<in data>, <out data>, <length>, vtk.vtkCommunicator.MAX_OP)
>
>
> to get the max values. vtkCommunicator has the named constants for
> operations. The "global" communicator will be the one that Catalyst uses
> which may be a subcommunicator of MPI_COMM_WORLD. I'm not sure if mpi4py
> will be over the Catalyst MPI processes or all of them. Also, you may want
> to try "from paraview import vtk" instead of just "import vtk". Less VTK
> Python stuff should be imported then but it may be missing something you
> want. I don't use vtkMPI4PYCommunicator so I cannot say how well that
> should work for your needs.
>
>
>
>
> I have a "NameError: global name 'vtkMPI4PyCommunicator' is not defined"
> message
>
> In my earlier tests, I tried simply using the AllReduce() method from the
> controller, with vtkArrays, but I was stuck on finding the correct value
> for the last argument, which seems to be an integer describing the
> collective reduction operation (I need min and max). But I was unable to
> find any examples for this in Python, (whether Googling for examples or
> printing dict() and (help() of various Python objects).
> The documentation only specifies some methods may not be available in
> Python...
>
> Are there named constants describing those operations in the Python
> wrappers of vtkMPIController ?
> I assume I could find the integer values in the C++ documentation/sources,
> but I would rather have named constants for readability and in case of
> future changes.
> I guess using the vtkMPIController would slightly simplify the code and
> reduce dependancy to some external Python modules.
>
> Thanks for the help, and best regards,
>
> Yvan
>
> ----- Mail original -----
> De: "Andy Bauer" < andy.bauer at kitware.com >
> À: "Joachim Pouderoux" < joachim.pouderoux at kitware.com >
> Cc: "Yvan Fournier" < yvan.fournier at free.fr >, "ParaView" <
> paraview at paraview.org >
> Envoyé: Mardi 14 Février 2017 21:27:06
> Objet: Re: [Paraview] Automating contour isosurface values in parallel
> Catalyst Python script
>
>
>
>
>
>
> Hi Yvan,
>
> Another option is just to compute it yourself. The reason that the global
> min and max aren't know in Catalyst is that it requires communication but
> many times the value isn't needed so in order to make Catalyst more
> efficient it doesn't compute that value for you automatically. In the
> Python scripts you can import mpi4py or import vtk and use the
> vtkMultiProcessController to do the global operation. Note that for my way
> you'll need to call UpdatePipeline(datadescription.GetTime()) before
> getting the output and globally determining the value. Then you can set the
> isosurfaces values array with that information.
>
> If you try this out and have issues, please let us know and we can go from
> there.
>
> Cheers,
> Andy
>
>
>
> On Tue, Feb 14, 2017 at 3:02 PM, Joachim Pouderoux <
> joachim.pouderoux at kitware.com > wrote:
>
>
>
>
>
>
>
>
> Hi Yvan,
>
> Regarding MinMax: this filter is indeed not exposed by any standard
> Catalyst edition but it is
> easy to add. See for instance in ParaView/Catalyst/Editions/Extras/manifest.json
> how the
> PVExtractSelection filter is exposed (line 50 and 247).
> Just do the same with the MinMax filter (see proxy's declaration at
> filters.xml:~11725).
>
> Best,
> Joachim
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Joachim Pouderoux , PhD
>
>
> Technical Expert - Scientific Computing Team
> Kitware SAS
>
>
>
>
> 2017-02-13 17:31 GMT-04:00 Yvan Fournier < yvan.fournier at free.fr > :
>
>
> Hello,
>
> I'm in the process of migrating some VTK scripts in the Code_Saturne (code-
> saturne.org ) test suite to Catalyst Python scripts, and am having some
>
>
> difficulty automating some cases using contour plots:
>
> For time-dependent data, I do not know the value ranges in advance, and
> would
> like to query them so as to build an Isosurface values array spanning that
> range.
>
> I can manage to access the point and cell data arrays and ther ranges, but
> printing the ranges in parallel, it seems I have local values only, while
> I need
> a global range.
>
> I searched for examples in the documentation, on the Wiki, ..., and found
> an
> example for MinMax, but don't seem to ba able to access the correct type of
> Proxy under Catalyst (the default servermanager is not connected in the
> usual
> way.
>
> For scalar bar color look-up tables, I can manage with
> "coprocessor.WriteImages(datadescription, rescale_lookuptable=True)" in
> the
> coprocessing script, but for a more general case such as thar needed for
> contour
> plots, I've been running around in circles in the last hours trying to
> find and
> adapt a relevant example.
>
> Does anyone have a suggestion ? Thanks in advance,
>
> Yvan Fournier
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/
> opensource/opensource.html
>
> Please keep messages on-topic and check the ParaView Wiki at:
> http://paraview.org/Wiki/ParaView
>
> Search the list archives at: http://markmail.org/search/?q=ParaView
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/paraview
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/
> opensource/opensource.html
>
> Please keep messages on-topic and check the ParaView Wiki at:
> http://paraview.org/Wiki/ParaView
>
> Search the list archives at: http://markmail.org/search/?q=ParaView
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/paraview
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview/attachments/20170217/0dc03792/attachment.html>