[vtkusers] Different behavior of vtkXMLPUnstructuredGridWriter in VTK 7.1

Andrew Parker andrew.parker at cantab.net
Mon Feb 19 07:51:29 EST 2018 

I'm using the following in the serial code:

pwriter->SetNumberOfPieces((int)nparts);
pwriter->SetGhostLevel(1);
pwriter->SetStartPiece((int)part);
pwriter->SetEndPiece((int)part);

My writer function in which the above code is called is called "nparts"
times, where part goes from 0->(nparts-1).

For the dummy controller, and nparts=4, I have:

vtkSmartPointer<vtkDummyController> vtk_mpi_controller;
vtk_mpi_controller = vtkSmartPointer<vtkDummyController>::New();
vtk_mpi_controller->Initialize(&argc, &argv, 1);
vtk_mpi_controller->SetGlobalController(vtk_mpi_controller);
vtk_mpi_controller->SetNumberOfProcesses(4);
....
vtk_mpi_controller->Finalize(1);

resulting bottom half of the pvtu reads:

    </PCellData>
    <PPoints>
      <PDataArray type="Float64" Name="Points" NumberOfComponents="3"/>
    </PPoints>
    <Piece Source="input_3.vtu"/>
  </PUnstructuredGrid>
</VTKFile>

Any thoughts?
Cheers,
Andy


On 19 February 2018 at 12:38, Andy Bauer <andy.bauer at kitware.com> wrote:

> If you haven't used the SetStartPiece(), SetEndPiece() and
> SetNumberOfPieces() methods in serial that's what is probably missing. In
> serial you should be able to use vtkDummyController.
>
>
>
> On Mon, Feb 19, 2018 at 4:59 AM, Andrew Parker <
> andy.john.parker at googlemail.com> wrote:
>
>> Andy,
>>
>> I can confirm this now works for my parallel code.  However, I have a
>> separate stand-alone serial code (has never had a communicator) which still
>> calls the vtkXMLPUnstructuredGridWriter and produces several part/piece
>> meshes and one global pvtu file.  This now fails with 7.1.1, same error
>> before, only one entry in the pvtu.  I have tried to make the changes noted
>> in your demo file (which have worked for my parallel code) but in this
>> instance they do not work in my serial code.  I have also tried the
>> vtkDummyController as a well as vtkmpicontroller, and have also tried
>> explicitly setting the number of (fake) processors: vtk_mpi_controller->
>> SetNumberOfProcesses(4);
>>
>> Could I ask what the recommended way to now use the vtkXMLPUnstructuredGridWriter
>> is from a completely serial code where I still wish to write out piece
>> meshes (and a corresponding pvtu) and do not have a communicator?  I should
>> add this is not threaded code either.  The vtk 7.1.1 build is the same one
>> that is now working correctly in my mpi parallel code.
>>
>> Cheers again,
>> Andy
>>
>> On 6 February 2018 at 13:48, Andy Bauer <andy.bauer at kitware.com> wrote:
>>
>>> My guess is that you've forgotten:
>>>
>>> controller->SetGlobalController(controller);
>>>
>>> Could you also verify that VTK was built with MPI enabled? If you try my
>>> example file does that work properly for you with both VTK versions?
>>>
>>>
>>> On Tue, Feb 6, 2018 at 4:49 AM, Andrew Parker <
>>> andy.john.parker at googlemail.com> wrote:
>>>
>>>> Andy,
>>>>
>>>> To follow up.  So this is a direct copy of my source for writing vtu in
>>>> parallel via mpi, no edits:
>>>>
>>>> vtkSmartPointer<vtkXMLPUnstructuredGridWriter> pwriter =
>>>> vtkSmartPointer<vtkXMLPUnstructuredGridWriter>::New();
>>>> std::stringstream sss;
>>>> sss << outputDirectory << "/";
>>>> sss << outputPrefix;
>>>> sss << "Parts";
>>>> sss << ".pvtu";
>>>>
>>>> pwriter->SetFileName(sss.str().c_str());
>>>> pwriter->SetNumberOfPieces(comms.Size());
>>>> if(comms.IsParallel())
>>>>   pwriter->SetGhostLevel(1);
>>>> pwriter->SetStartPiece(comms.Rank());
>>>> pwriter->SetEndPiece(comms.Rank());
>>>> pwriter->SetInputData(cellGrid);
>>>> // pwriter->SetDataModeToAscii();
>>>> pwriter->SetDataModeToBinary();
>>>> pwriter->SetCompressorTypeToZLib();
>>>> pwriter->Write();
>>>>
>>>> and for vtp files in parallel via mpi, no edits:
>>>>
>>>> vtkSmartPointer<vtkXMLPPolyDataWriter> pwriter =
>>>> vtkSmartPointer<vtkXMLPPolyDataWriter>::New();
>>>> std::stringstream sss;
>>>> sss << outputDirectory << "/";
>>>> sss << outputPrefix;
>>>> sss << bcName << "_";
>>>> sss << prefix;
>>>> sss << ".pvtp";
>>>> std::cout << "Output file name: " << sss.str().c_str() << std::endl;
>>>>
>>>> pwriter->SetFileName(sss.str().c_str());
>>>> pwriter->SetNumberOfPieces(comms->Size());
>>>> if(comms->IsParallel())
>>>>   pwriter->SetGhostLevel(1);
>>>> pwriter->SetStartPiece(comms->Rank());
>>>> pwriter->SetEndPiece(comms->Rank());
>>>> pwriter->SetInputData(boundingFaceGrids[i]);
>>>> pwriter->SetDataModeToBinary();
>>>> //pwriter->SetDataModeToAscii();
>>>> pwriter->SetCompressorTypeToZLib();
>>>> pwriter->Write();
>>>>
>>>> I initialise the vtk-comms object in an identical way to your demo
>>>> code, after my own mpi-comms, and I finalize before I finalize mine.  In
>>>> between those calls the above source is called.  All processors partake in
>>>> all calls both to init the vtk-mpi and in the above writing.
>>>>
>>>> Do you have any further thoughts as to the problem?  Problem goes away
>>>> if I use 6.3....
>>>>
>>>> Cheers,
>>>> Andy
>>>>
>>>>
>>>> On 5 February 2018 at 18:40, Andrew Parker <
>>>> andy.john.parker at googlemail.com> wrote:
>>>>
>>>>> Andy,
>>>>>
>>>>> My own code. I’ve always had those Piece lines in my src from day one,
>>>>> and always used it via mpi, so don’t need added. All procs call all lines
>>>>> in my file writer. There is no if-master-section.  Literally just a vtk
>>>>> version number change that causes the problem. Can’t get the data files or
>>>>> the cut down source until tomorrow.
>>>>>
>>>>> I shared the other two links as those in addition to this post are the
>>>>> only places I can find that reference such a major change to the library
>>>>> that is undocumented and causes such a significant change to user code. Are
>>>>> there any further references on this change that I can read about to try to
>>>>> resolve this?
>>>>>
>>>>> I find it strange that this has not come up more. It would mean all
>>>>> users of 7.1 without the additional vtkmpicontroller lines would fail when
>>>>> calling a parallel writer from a parallel mpi-user-code....
>>>>>
>>>>> Cheers,
>>>>> Andy
>>>>>
>>>>> On Mon, 5 Feb 2018 at 18:07, Andy Bauer <andy.bauer at kitware.com>
>>>>> wrote:
>>>>>
>>>>>> Are you using libMesh or another simulation code? You'll definitely
>>>>>> need the following to tell each process what piece it has:
>>>>>> pwriter->SetNumberOfPieces(nprocs);
>>>>>>
>>>>>> pwriter->SetStartPiece(rank);
>>>>>>
>>>>>>
>>>>>> pwriter->SetEndPiece(rank);
>>>>>>
>>>>>>
>>>>>> pwriter->Write();
>>>>>>
>>>>>> Can you share the portion of your code that uses
>>>>>> vtkXMLPUnstructuredGridWriter? Also, looking at the data files from 6.3 and
>>>>>> 7.1.1 may help.
>>>>>>
>>>>>> Thanks,
>>>>>> Andy
>>>>>>
>>>>>>
>>>>>> On Mon, Feb 5, 2018 at 12:08 PM, Andrew Parker <
>>>>>> andy.john.parker at googlemail.com> wrote:
>>>>>>
>>>>>>> Andy,
>>>>>>>
>>>>>>> Just moved from vtk 6.3.0 to vtk 7.1.1.  Cannot for the life of me
>>>>>>> get your modifications to work from your vtk_test.C.  I've included the
>>>>>>> vtkmpiController stuff after I init mpi myself.  I found the migration
>>>>>>> problem mentioned here:
>>>>>>>
>>>>>>> https://github.com/libMesh/libmesh/issues/1179
>>>>>>>
>>>>>>> and here:
>>>>>>>
>>>>>>> https://gitlab.kitware.com/vtk/vtk/issues/16924
>>>>>>>
>>>>>>> as well as this post.  My parallel unstructured grid now only
>>>>>>> contains one source within the piece section of the pvtu, as opposed to
>>>>>>> n-processor sources (all processors have real data to write and this is not
>>>>>>> a case of empty meshes on these processors).  I can re-link and recompile
>>>>>>> against 6.3 and the problem goes away and I get the correct number of
>>>>>>> sources written to the pvtu file as before.  Can you help?  I make
>>>>>>> extensive use of pvtu files from my mpi-based code and now all grid files
>>>>>>> (pvtu and pvtp) have stopped working just by moving to 7.1.1 (which has
>>>>>>> been built with mpi-enabled).
>>>>>>>
>>>>>>> Any thoughts?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Andy
>>>>>>>
>>>>>>>
>>>>>>> On 6 December 2016 at 16:12, Andy Bauer <andy.bauer at kitware.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi John,
>>>>>>>>
>>>>>>>> The issue is indeed due to changes that I mentioned before (i.e.
>>>>>>>> only data sets that have data get written out and the .pvtu file is
>>>>>>>> adjusted accordingly). With this change though interprocess communication
>>>>>>>> is required which is done the the vtkMultiProcessController::GetGlobalController()
>>>>>>>> object which essentially acts as MPI_COMM_WORLD within VTK. You need to
>>>>>>>> create a vtkMPIController which derives from vtkMultiProcessController. See
>>>>>>>> the attached files for how that is done. This should also be backwards
>>>>>>>> compatible but if it isn't, please let us know.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Andy
>>>>>>>>
>>>>>>>> ps. Ignore the ADIOS stuff in there (it's commented out). I was
>>>>>>>> trying to do two things at once and that's why the ADIOS stuff is in the
>>>>>>>> CMakeLists.txt.
>>>>>>>>
>>>>>>>> On Mon, Dec 5, 2016 at 6:06 PM, John Peterson <jwpeterson at gmail.com
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Dec 5, 2016 at 12:45 PM, Andy Bauer <
>>>>>>>>> andy.bauer at kitware.com> wrote:
>>>>>>>>>
>>>>>>>>>> Beyond this I can't think of any change off the top of my head
>>>>>>>>>> that would cause your issue. I looked at the repo but can't figure out the
>>>>>>>>>> VTK configurations from that. If you can share a simple test case which
>>>>>>>>>> demonstrates the issue I may be able to easily diagnose it.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Here's a standalone test code that reproduces the issue for me:
>>>>>>>>>
>>>>>>>>> https://gist.github.com/jwpeterson/92f4b8422d6fbb1056e848c9b
>>>>>>>>> 14ee1d7
>>>>>>>>>
>>>>>>>>> Run on two procs with: mpiexec -np 2 and I get the following
>>>>>>>>> test.pvtu file:
>>>>>>>>>
>>>>>>>>> <?xml version="1.0"?>
>>>>>>>>>> <VTKFile type="PUnstructuredGrid" version="0.1"
>>>>>>>>>> byte_order="LittleEndian" header_type="UInt32"
>>>>>>>>>> compressor="vtkZLibDataCompressor">
>>>>>>>>>>   <PUnstructuredGrid GhostLevel="1">
>>>>>>>>>>     <PPointData>
>>>>>>>>>>       <PDataArray type="Float64" Name="u"/>
>>>>>>>>>>     </PPointData>
>>>>>>>>>>     <PCellData>
>>>>>>>>>>       <PDataArray type="Int32" Name="elem_id"/>
>>>>>>>>>>       <PDataArray type="Int32" Name="subdomain_id"/>
>>>>>>>>>>       <PDataArray type="Int32" Name="processor_id"/>
>>>>>>>>>>     </PCellData>
>>>>>>>>>>     <PPoints>
>>>>>>>>>>       <PDataArray type="Float64" Name="Points"
>>>>>>>>>> NumberOfComponents="3"/>
>>>>>>>>>>     </PPoints>
>>>>>>>>>>     <Piece Source="test_0.vtu"/>
>>>>>>>>>>   </PUnstructuredGrid>
>>>>>>>>>> </VTKFile>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> As you can see, only one "Piece Source" is listed, but the output
>>>>>>>>> of the program is actually all 3 files (test.pvtu, test_0.vtu, test_1.vtu)
>>>>>>>>> and both the _0 and _1 files have nodes in them.  Furthermore, if I simply
>>>>>>>>> add a second "Piece Source" line corresponding to "test_1.vtu", the whole
>>>>>>>>> thing opens up just fine in Paraview.  I am a novice VTK programmer so it's
>>>>>>>>> very possible I'm doing something wrong, but this code definitely worked in
>>>>>>>>> VTK 6.x, and 7.0.
>>>>>>>>>
>>>>>>>>> Thanks for your help,
>>>>>>>>> John
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Powered by www.kitware.com
>>>>>>>>
>>>>>>>> Visit other Kitware open-source projects at
>>>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>>>
>>>>>>>> Please keep messages on-topic and check the VTK FAQ at:
>>>>>>>> http://www.vtk.org/Wiki/VTK_FAQ
>>>>>>>>
>>>>>>>> Search the list archives at: http://markmail.org/search/?q=vtkusers
>>>>>>>>
>>>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>>>> http://public.kitware.com/mailman/listinfo/vtkusers
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://vtk.org/pipermail/vtkusers/attachments/20180219/e9278ff1/attachment.html>

More information about the vtkusers mailing list