[Paraview] pvbatch, MPI and MultiBlock data sets

[Utkarsh Ayachit]

Actually this was bug that fix just a couple of days ago:
http://paraview.org/Bug/view.php?id=13850
It should be fixed in the upcoming release (3.98.2).

Utkarsh

On Mon, Feb 4, 2013 at 4:05 PM, Yves Rogez
<yves.rogez at obs.ujf-grenoble.fr> wrote:
> Hi,
>
> I found a solution to this problem. I had to insert an UpdatePipeline() call
> just before the writer update :
>
> myLastFilter.UpdatePipeline()
> writer.UpdatePipeline()
>
> This is a strange behavior. Maybe due to MPICH2 usage ? I don't know...
> However it runs well like that.
>
> Thanks for your help
>
> Yves
>
>
> Le mardi 29 janvier 2013 15:56:32, Utkarsh Ayachit a écrit :
>
>
> Yves,
>
> 1. I don't see the oddity with the block number and BlockID when
> running in parallel, as you describe. For me, the BlockID in indeed
> Index for block + 1 whether I run in parlalel or not.
> 2. Running your python script as a single process, I get 1:18 minutes,
> while in parallel (2 procs), I get 1:22 mins. Given that your dataset
> very small and the computation done also very trivial (you are
> transforming 50 points!), it's not surprising that the overheads of
> parallel processing (interprocessing communication for various steps
> in the pipeline execution) make things slower. Try doing the same with
> a multiblock having thousands of points in each block and then you'll
> see the different
>
> Utkarsh
>
>
>
> On Tue, Jan 15, 2013 at 11:17 AM, Yves Rogez
> <yves.rogez at obs.ujf-grenoble.fr> wrote:
>
>
> Ok, trying ProcessIdScalars filter obliged me to see the results more in
> details (with spreadsheet view).
>
> My input : 50 blocks containing each 1 polydata constituted by 1 point and 1
> vertex cell
>
> With a single process :
> I get 50 blocks containing 1 point and 1 vertex cell, with PID = 0,
> block number = ( index of the block + 1 )
> So it is OK
>
> With MPI (2 processes) :
> I get 50 blocks containing 2 points and 2 vertices cells, each with
> strange block numbers :
> block id 0 -> pt 1 has BN=2, pt 2 has BN=3
> block id 1 -> BN=(5,6)
> block id 2 -> BN=(8,9)
> block id 3 -> BN=(11,12) and so on...
> It seems :
> BN for Point1 = ( ( BlockID + 1 ) * 3 ) - 1
> BN for Point2 = ( BlockID + 1 ) * 3
>
> Point 1 of a block has always PID=0 and point 2 PID=1
>
> Maybe I made something wrong when generating my input (please find attached
> a zip) ?
>
> Yves Rogez
>
> IPAG
> Institut de Planétologie et d'Astrophysique de Grenoble
> Bat D de Physique - BP. 53 - 38041 Grenoble - FRANCE
>
> tel : +33 (0)4 76 63 52 80
> lab : +33 (0)4 76 63 52 89
> Le 15/01/2013 16:25, Utkarsh Ayachit a écrit :
>
> Just to make sure, your ParaView is built with MPI support enabled,
> right? XMLMultiBlockDataReader does distribute the blocks to read
> among the processes. Try apply a "ProcessIdScalars" filter in the
> middle and then look at the ProcessId assigned to the blocks in the
> data. They should show how the blocks were distributed.
>
> Utkarsh
>
> On Tue, Jan 15, 2013 at 7:20 AM, Yves Rogez
> <yves.rogez at obs.ujf-grenoble.fr> wrote:
>
> Hello,
>
> I'm trying to parallelize a process using pvbatch and MPI, with MultiBlock
> data set; thus using the vtk composite pipeline.
> I made a sample python program that is representative of what I have to do :
>
> --------------------------------------------------------------------------------------------------
>
> from paraview.simple import *
>
> r = servermanager.sources.XMLMultiBlockDataReader()
> r.FileName = "input.vtm"
>
> # Defining a sample fake data processing
> nbTs = 1000
> ts = {}
> for tIndex in range( 0, nbTs ):
> ts[tIndex] = servermanager.filters.Transform()
> if tIndex == 0:
> ts[tIndex].Input = r
> else:
> ts[tIndex].Input = ts[tIndex - 1]
> ts[tIndex].Transform.Scale = [1.01,1.01,1.01]
>
> w = servermanager.writers.XMLMultiBlockDataWriter()
> w.Input = ts[nbTs - 1]
> w.FileName = "output.vtm"
>
> w.UpdatePipeline()
>
> --------------------------------------------------------------------------------------------------
>
> I launch that using "mpiexec -np 4 pvbatch myscript.py"
> All run well but I get a longer time using MPI than using only "pvbatch
> myscript.py".
>
> By monitoring RAM, I noticed that it seems the data is loaded on time by MPI
> process, and (maybe) all the MPI processes do exactly the same job,
> computing four times all the data.
>
> Why my blocks in MultiBlock data set aren't dispatched over the MPI
> processes ?
> What am I doing wrong ?
>
> Many thanks for any help,
>
> Yves
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the ParaView Wiki at:
> http://paraview.org/Wiki/ParaView
>
> Follow this link to subscribe/unsubscribe:
> http://www.paraview.org/mailman/listinfo/paraview
>
>
> --
> Yves Rogez
> IPAG / CNRS