[vtkusers] vtk.vtkDistributedDataFilter() not works

[Cory Quammen]

Magician,

I am transitioning this thread over to the ParaView users mailing list
as you are using pvpython, and this is actually simpler to do using
the paraview.simple module rather than with VTK directly.

What you are trying to do is not going to work with pvpython. pvpython
is a client that you can use with either the built-in server or by
connecting to a remote server, much like the ParaView application. You
won't be able to run it in parallel and run parallel VTK code with it.

Instead, you will need to invoke pvpython with a script that connects
to a running pvserver with 2 processors.

Here's something that worked for me for partitioning a point source:

* Launch pvserver with two processes (mpirun -np 2 pvserver)
* Launch pvpython script.py
* The contents of script.py should be:

from paraview.simple import *

Connect("localhost")

pointSource = PointSource()
pointSource.NumberOfPoints = 10000
pointSource.Radius = 1.0

glyph = Glyph(Input=pointSource, GlyphType='Arrow')
glyph.GlyphType = '2D Glyph'
glyph.GlyphType.GlyphType = 'Vertex'
glyph.GlyphMode = 'All Points'

d3 = D3(Input=glyph)

SaveData('tmp.pvtu', proxy=d3)

The Glyph filter is needed to add vertex cells at each point. D3 needs
these cells to do the partitioning properly.

Can you get this script to work?

Thanks,
Cory

On Mon, Dec 19, 2016 at 4:54 PM, Magician <f_magician at mac.com> wrote:
> Hi Cory,
>
>
> Sorry again. (My mailer has some trouble...)
>
> I tried Sphere Source too, but the results are same.
>
>> src = vtk.vtkSphereSource()
>> src.SetCenter(0.0, 0.0, 0.0)
>> src.SetRadius(1.0)
>> src.SetThetaResolution(8)
>> src.SetPhiResolution(5)
>> src.Update()
>
> If the code runs on native VTK environments, maybe my ParaView’s VTK has some trouble.
>
>
> Magician
>
>
>> On Dec 13, 2016, at 12:03, Cory Quammen <cory.quammen at kitware.com> wrote:
>>
>> Hmm, I'm not sure. Could you try setting the input to d3 to the output
>> of a vtkSphereSource and see if you get the expected results there?
>> You should get half the sphere in one piece and the other half sphere
>> in the other piece.
>>
>> Thanks,
>> Cory
>>
>> On Sat, Dec 10, 2016 at 10:35 AM, Magician <f_magician at mac.com> wrote:
>>> Hi Cory,
>>>
>>>
>>> Sorry for my late reply.
>>> I modified my code for splitting the points, but the filter doesn’t work.
>>>
>>>
>>>    import vtk
>>>
>>>    src = vtk.vtkPointSource()
>>>    src.SetCenter((0.0, 0.0, 0.0))
>>>    src.SetNumberOfPoints(1000)
>>>    src.SetRadius(1.0)
>>>    src.Update()
>>>
>>>    dst = vtk.vtkPolyData()
>>>    pts = vtk.vtkPoints()
>>>    pts.SetData(src.GetOutput().GetPoints().GetData())
>>>    dst.SetPoints(pts)
>>>    cells = vtk.vtkCellArray()
>>>    for i in range(src.GetOutput().GetNumberOfPoints()):
>>>        ptIds = vtk.vtkIdList()
>>>        ptIds.InsertNextId(i)
>>>        cells.InsertNextCell(ptIds)
>>>    dst.SetVerts(cells)
>>>
>>>    d3 = vtk.vtkDistributedDataFilter()
>>>    d3.UseMinimalMemoryOff()
>>>    d3.SetInputData(dst)
>>>    d3.SetBoundaryMode(0)
>>>    d3.SetBoundaryModeToSplitBoundaryCells()
>>>    d3.Update()
>>>
>>>    writer = vtk.vtkXMLPUnstructuredGridWriter()
>>>    writer.SetInputData(d3.GetOutput())
>>>    writer.SetFileName(’test.pvtu’)
>>>    writer.SetNumberOfPieces(2)
>>>    writer.WriteSummaryFileOn()
>>>    writer.SetStartPiece(0)
>>>    writer.SetEndPiece(1)
>>>    writer.Write()
>>>
>>>
>>> Magician
>>>
>>>
>>>> On Dec 6, 2016, at 01:33, Cory Quammen <cory.quammen at kitware.com> wrote:
>>>>
>>>> I'm not 100% sure of this, but the vtkPointSource produces an output
>>>> with only 1 cell. That may not be partitionable by the
>>>> vtkDistributeDataFilter. You shouldn't get duplicate output in each
>>>> piece, but let's not worry about that for now.
>>>>
>>>> Try swapping out the vtkPointSource with a vtkSphereSource and see if
>>>> the data is partitioned.
>>>>
>>>> HTH,
>>>> Cory
>>>>
>>>> On Sat, Dec 3, 2016 at 8:57 PM, Magician <f_magician at mac.com> wrote:
>>>>> Hi Cory,
>>>>>
>>>>>
>>>>> Thanks for your advice.
>>>>>
>>>>> Hmmm...I already run the script with MPI.
>>>>> The attached script is a minimal sample.
>>>>>
>>>>>
>>>>> If I execute it, pvtu file is exported.
>>>>> But the all piece sources are exactly same, and not partitioned.
>>>>>
>>>>>
>>>>>
>>>>> Magician
>>>>>
>>>>>
>>>>>> On Nov 28, 2016, at 06:19, Cory Quammen <cory.quammen at kitware.com> wrote:
>>>>>>
>>>>>> I think you need to run your script in parallel with MPI for the
>>>>>> partitioning to work. See [1] for an example of how to use this
>>>>>> filter.
>>>>>>
>>>>>> Hope that helps,
>>>>>> Cory
>>>>>>
>>>>>> [1] http://www.vtk.org/gitweb?p=VTK.git;a=blob;f=Filters/Parallel/Testing/Cxx/DistributedData.cxx
>>>>>>
>>>>>> On Sun, Nov 27, 2016 at 10:39 AM, Magician <f_magician at mac.com> wrote:
>>>>>>> Does anyone use the vtkDistributedDataFilter?
>>>>>>>
>>>>>>>
>>>>>>> On Nov 19, 2016, at 18:16, Magician <f_magician at mac.com> wrote:
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>>
>>>>>>> I posted the message about partitioning datasets.
>>>>>>> <http://public.kitware.com/pipermail/vtkusers/2016-November/097123.html>
>>>>>>>
>>>>>>> I still trying vtk.vtkDistributedDataFilter(), but the exported data aren’t
>>>>>>> partitioned.
>>>>>>> My VTK version is 7.1.0 (pvpython with ParaView 5.2.0 RC3) and executing on
>>>>>>> 2 CPUs.
>>>>>>>
>>>>>>> Here is the sample code:
>>>>>>>
>>>>>>>  import vtk
>>>>>>>
>>>>>>>  source = vtk.vtkPointSource()
>>>>>>>  source.SetCenter((0.0, 0.0, 0.0))
>>>>>>>  source.SetNumberOfPoints(1000000)
>>>>>>>  source.SetRadius(1.0)
>>>>>>>  source.Update()
>>>>>>>
>>>>>>>  d3 = vtk.vtkDistributedDataFilter()
>>>>>>>  d3.SetInputData(source.GetOutput())
>>>>>>>  d3.SetBoundaryMode(0)
>>>>>>>  d3.Update()
>>>>>>>
>>>>>>>  writer = vtk.vtkXMLPUnstructuredGridWriter()
>>>>>>>  writer.SetInputData(d3.GetOutput())
>>>>>>>  writer.SetFileName(’test.pvtu’)
>>>>>>>  writer.SetNumberOfPieces(2)
>>>>>>>  writer.WriteSummaryFileOn()
>>>>>>>  writer.SetStartPiece(0)
>>>>>>>  writer.SetEndPiece(1)
>>>>>>>  writer.Write()
>>>>>>>
>>>>>>> How to partition data?
>>>>>>>
>>>>>>>
>>>>>>> Magician
>



-- 
Cory Quammen
Staff R&D Engineer
Kitware, Inc.