[Paraview] [vtkusers] vtkDistributedDataFilter and ghost cells
Paul Edwards
paul.m.edwards at gmail.com
Thu Nov 8 11:22:28 EST 2012
What I have done is used D3 internally in my reader to partition the data
and always have just 1 ghost level. This could mean issues if more were
needed but it's not necessary for the usual post-processing tasks of users.
To add global ids just create the array and call SetGlobalIds on your
dataset.
Regards,
Paul
On 8 November 2012 09:43, Andrew Parker <andy.john.parker at googlemail.com>wrote:
> Hi mate, how goes it?
>
> How do you set the ghost level to be one rather than default zero then?
> Have you written your own "downstream" filter to update the number of
> ghost levels? If so any tips?
>
> "If you know the name of a global node ID array in the input dataset, set
> that name with this method"... Which method?? How do I set this? I could
> probably work out the global node and cell numbers and stick them into the
> meshes that I pass to D3. But how do I do that? What method do I call to
> set them, and what name do I give them.... "this method" ain't that helpful
> in the docs ;-)
>
> Cheers,
> Andy
>
> On 8 November 2012 01:18, Paul Edwards <paul.m.edwards at gmail.com> wrote:
>
>> Hi Andy,
>>
>> I've had a few problems with D3. The limitation I find is that global
>> node ids are slow to generate and the point merging (I think) is only
>> performed in single precision. Also, remember that everything will be
>> redone each time the number of ghost levels that are requested changes -
>> this means the global ids are recalculated, it is repartitioned and then
>> the number of ghost levels are added. In some ways it might be better to
>> have D3 split into 3 filters for the different steps although it would make
>> a long pipeline.
>>
>> Btw looking at the docs it seems to suggest that it only creates
>> temporary arrays for global ids within d3 (although I think in the past I
>> have had access to the global ids it created):
>>
>> If you know the name of a global node ID array in the input dataset,
>> set that name with this method. If you leave it unset, D3 will search the
>> input data set for certain common names of global node ID arrays. If none
>> is found, and ghost cells have been requested,* D3 will create a
>> temporary global node ID array* before acquiring ghost cells. It is also
>> desirable to have global element IDs. However, if they don't exist D3 can
>> create them relatively quickly. Set the name of the global element ID array
>> if you have it. If it is not set, D3 will search for it using common names.
>> If still not found, D3 will create a temporary array of global element IDs.
>>
>>
>> My filters now add their own global ids so I don't use d3 for that.
>>
>> Regards,
>> Paul
>>
>>
>> On 7 November 2012 14:50, George Zagaris <george.zagaris at kitware.com>wrote:
>>
>>> On Wed, Nov 7, 2012 at 9:34 AM, Andrew Parker
>>> <andy.john.parker at googlemail.com> wrote:
>>> > Many thanks for that. I perform some threshold stuff early on, so
>>> basically
>>> >
>>>
>>> Perhaps an easy solution is to append a "GlobalCellID" and
>>> "GlobalNodeID" before thresholding. To the best of my knowledge,
>>> threshold will extract vtkUnstructuredGrid instance from whatever goes
>>> in the input and not automatically retain this type of global
>>> information.
>>>
>>> Hope this helps.
>>>
>>> Best,
>>> George
>>>
>>> > On 7 November 2012 13:00, George Zagaris <george.zagaris at kitware.com>
>>> wrote:
>>> >>
>>> >> Hello,
>>> >>
>>> >> Since the data is essentially structured (rectilinear grids), I think
>>> >> the best way to get the global/local mapping is to use extent
>>> >> information. Specifically, you need the whole extent of the grid and
>>> >> the global extent of each partition. Then from the global IJK of a
>>> >> point or cell, and the whole extent you can use methods from
>>> >> vtkStructuredData to compute the corresponding global linear index for
>>> >> a point or cell. Do you have extent information?
>>> >>
>>> >> Also, I have done some work on ghost generation in VTK for structured
>>> >> (and AMR) data. Unfortunately, currently it doesn't work outside the
>>> >> box with rectilinear grids. This functionality can still be used to
>>> >> create ghost layers and exchange the fields (node and/or cell data) on
>>> >> the grid but, it doesn't communicate the coordindates on the
>>> >> rectilinear grid. Alternatively, you may be able to converge to a
>>> >> solution faster if you convert the rectilinear grid to a
>>> >> vtkStructuredGrid and use vtkPStructuredGridConnectivity or
>>> >> vtkPStructuredGridGhostDataGenerator to generate ghost layers. There
>>> >> are tests that illustrate how to use these classes and I can provide
>>> >> more information if you are interested.
>>> >>
>>> >> Best,
>>> >> George
>>> >>
>>> >> On Wed, Nov 7, 2012 at 7:22 AM, Moreland, Kenneth <kmorel at sandia.gov>
>>> >> wrote:
>>> >> > Hmmm. I thought global point IDs were always created. Maybe it only
>>> >> > happens
>>> >> > when ghost cells are created. Try getting that working first.
>>> >> >
>>> >> > As far as global cell IDs are concerned, you are out of luck.
>>> >> > vtkDistributedDataFilter does not create them. But they are not all
>>> that
>>> >> > useful anyway since, unlike points, once you remove the out cells
>>> there
>>> >> > is
>>> >> > no overlap. Usually you can get all the information you want from
>>> the
>>> >> > ghost
>>> >> > level.
>>> >> >
>>> >> > If you really need global cell IDs, I'm sure you would have no
>>> trouble
>>> >> > making them yourself. I would do it before running
>>> >> > vtkDistributedDataFilter.
>>> >> > Assuming when you first read in your data you have no duplicate
>>> cells,
>>> >> > just
>>> >> > label those on process 0 as 0..N_0, those on process 1 as
>>> N_0+1..N_1,
>>> >> > and so
>>> >> > on.
>>> >> >
>>> >> > -Ken
>>> >> >
>>> >> > Sent from my iPad so blame autocorrect.
>>> >> >
>>> >> > On Nov 7, 2012, at 2:38 AM, "Andrew Parker"
>>> >> > <andy.john.parker at googlemail.com> wrote:
>>> >> >
>>> >> > So that's working now in terms of the cast, forgot GetOutput()
>>> inside
>>> >> > the
>>> >> > cast operator! The returned vtkUGrid is now fully populated and
>>> >> > contains
>>> >> > sensible information.
>>> >> >
>>> >> > However, both getPointData()->GetGlobalIds() and
>>> >> > getCellData()->GetGlobalIds() return null pointers. Any thoughts?
>>> >> >
>>> >> > Also, should I be using CellData since I want the cell global to
>>> local
>>> >> > mapping for cells not the nodes, at the moment at least?
>>> >> >
>>> >> > On 6 November 2012 19:05, Moreland, Kenneth <kmorel at sandia.gov>
>>> wrote:
>>> >> >>
>>> >> >> Perhaps it is outputting a composite data set of some type? Try
>>> >> >> running
>>> >> >> GetClassName() to see what type of data object it really is.
>>> >> >>
>>> >> >> -Ken
>>> >> >>
>>> >> >> From: Andrew Parker <andy.john.parker at googlemail.com>
>>> >> >> Date: Tuesday, November 6, 2012 9:50 AM
>>> >> >>
>>> >> >> To: Kenneth Moreland <kmorel at sandia.gov>
>>> >> >> Cc: "vtkusers at vtk.org" <vtkusers at vtk.org>, "paraview at paraview.org"
>>> >> >> <paraview at paraview.org>
>>> >> >> Subject: [EXTERNAL] Re: [Paraview] vtkDistributedDataFilter and
>>> ghost
>>> >> >> cells
>>> >> >>
>>> >> >> Thanks on both accounts. Any thoughts why the downcast called
>>> after
>>> >> >> dd->Update() on distributedDataFilter is a null pointer? As in,
>>> dd is
>>> >> >> working perfectly properly, but I don't seem to be able to extract
>>> a
>>> >> >> valid
>>> >> >> unstructuredgrid. For a follow up question I assume
>>> >> >> getPointData()->GetGlobalIds() gives the local to global for the
>>> mesh
>>> >> >> nodes,
>>> >> >> and I should use getCellData()->GetGlobalIds() to get the local to
>>> >> >> global
>>> >> >> for the cells? Once I get a valid pointer that is....
>>> >> >>
>>> >> >> Cheers again,
>>> >> >> Andy
>>> >> >>
>>> >> >> On 6 November 2012 16:40, Moreland, Kenneth <kmorel at sandia.gov>
>>> wrote:
>>> >> >>>
>>> >> >>> You should be able to do a vtkUnstructuredGrid::SafeDownCast() to
>>> the
>>> >> >>> data to get the unstructured mesh (and access to the point data).
>>> >> >>>
>>> >> >>> -Ken
>>> >> >>>
>>> >> >>> From: Andrew Parker <andy.john.parker at googlemail.com>
>>> >> >>> Date: Tuesday, November 6, 2012 9:32 AM
>>> >> >>> To: Kenneth Moreland <kmorel at sandia.gov>
>>> >> >>> Cc: "vtkusers at vtk.org" <vtkusers at vtk.org>, "paraview at paraview.org
>>> "
>>> >> >>> <paraview at paraview.org>
>>> >> >>> Subject: [EXTERNAL] Re: [Paraview] vtkDistributedDataFilter and
>>> ghost
>>> >> >>> cells
>>> >> >>>
>>> >> >>> Another question which I'd forgotten, how do I get to a
>>> >> >>> vtkUnstructuredGrid per processor from the
>>> vtkDistributedDataFilter.
>>> >> >>>
>>> >> >>> For instance, dd->GetOutput()->GetPointData()->GetGlobalIds()
>>> doesn't
>>> >> >>> work as it's a vtkDataObject
>>> >> >>>
>>> >> >>> Stupid question I'm sure but the doxy notes say this type returns
>>> an
>>> >> >>> unstructured mesh, but I can't seem to get it out?
>>> >> >>>
>>> >> >>> Also, why exactly do I need the vtkPieceScalars and
>>> >> >>> vtkDataSetSurfaceFilter again? If the above can be made to work
>>> and
>>> >> >>> return
>>> >> >>> the mapping, what are they adding in terms of information?
>>> >> >>>
>>> >> >>> Thanks again,
>>> >> >>> Andy
>>> >> >>>
>>> >> >>> On 6 November 2012 16:00, Moreland, Kenneth <kmorel at sandia.gov>
>>> wrote:
>>> >> >>>>
>>> >> >>>> I believe vtkDistributedDataFilter will always return with global
>>> >> >>>> point
>>> >> >>>> ids (a mapping from local point ids to global point ids),
>>> although it
>>> >> >>>> might
>>> >> >>>> pass them if they already exist. So
>>> >> >>>> dd->GetOutput()->GetPointData()->GetGlobalIds() should return the
>>> >> >>>> array that
>>> >> >>>> gives this mapping.
>>> >> >>>>
>>> >> >>>> Ghost cells are only created on demand, and this is usually done
>>> by
>>> >> >>>> pipeline convention. If you have a filter that needs a layer of
>>> >> >>>> ghost
>>> >> >>>> cells, it should override the RequestUpdateExtent method to
>>> increment
>>> >> >>>> the
>>> >> >>>> value of UPDATE_NUMBER_OF_GHOST_LEVELS from the output
>>> information to
>>> >> >>>> the
>>> >> >>>> input information. This method would look something like this.
>>> >> >>>>
>>> >> >>>> int vtkDistributedDataFilter::RequestUpdateExtent(
>>> >> >>>> vtkInformation *vtkNotUsed(request),
>>> >> >>>> vtkInformationVector **inputVector,
>>> >> >>>> vtkInformationVector *outputVector)
>>> >> >>>> {
>>> >> >>>> // get the info objects
>>> >> >>>> vtkInformation *inInfo =
>>> inputVector[0]->GetInformationObject(0);
>>> >> >>>> vtkInformation *outInfo =
>>> outputVector->GetInformationObject(0);
>>> >> >>>>
>>> >> >>>> int piece, numPieces, ghostLevels;
>>> >> >>>>
>>> >> >>>> // We require an extra layer of ghost cells from upstream.
>>> >> >>>>
>>> >> >>>> piece =
>>> >> >>>>
>>> >> >>>>
>>> outInfo->Get(vtkStreamingDemandDrivenPipeline::UPDATE_PIECE_NUMBER());
>>> >> >>>> numPieces =
>>> >> >>>>
>>> >> >>>>
>>> >> >>>>
>>> outInfo->Get(vtkStreamingDemandDrivenPipeline::UPDATE_NUMBER_OF_PIECES());
>>> >> >>>> ghostLevels =
>>> >> >>>>
>>> >> >>>>
>>> >> >>>>
>>> outInfo->Get(vtkStreamingDemandDrivenPipeline::UPDATE_NUMBER_OF_GHOST_LEVELS());
>>> >> >>>>
>>> >> >>>>
>>> >> >>>>
>>> inInfo->Set(vtkStreamingDemandDrivenPipeline::UPDATE_PIECE_NUMBER(),
>>> >> >>>> piece);
>>> >> >>>>
>>> >> >>>>
>>> >> >>>>
>>> inInfo->Set(vtkStreamingDemandDrivenPipeline::UPDATE_NUMBER_OF_PIECES(),
>>> >> >>>> numPieces);
>>> >> >>>>
>>> >> >>>>
>>> >> >>>>
>>> inInfo->Set(vtkStreamingDemandDrivenPipeline::UPDATE_NUMBER_OF_GHOST_LEVELS(),
>>> >> >>>> ghostLevels);
>>> >> >>>>
>>> >> >>>> return 1;
>>> >> >>>> }
>>> >> >>>>
>>> >> >>>>
>>> >> >>>> The operation of the RequestData method should also strip off
>>> this
>>> >> >>>> layer
>>> >> >>>> of ghost cells. It might be possible to request a layer of ghost
>>> >> >>>> cells by
>>> >> >>>> setting UPDATE_NUMBER_OF_GHOST_LEVELS at the bottom of the
>>> pipeline,
>>> >> >>>> but I'm
>>> >> >>>> not totally sure how to make that work. It's probably easier
>>> (or at
>>> >> >>>> least
>>> >> >>>> cleaner) to do it from within a filter.
>>> >> >>>>
>>> >> >>>> -Ken
>>> >> >>>>
>>> >> >>>> From: Andrew Parker <andy.john.parker at googlemail.com>
>>> >> >>>> Date: Tuesday, November 6, 2012 8:25 AM
>>> >> >>>> To: "vtkusers at vtk.org" <vtkusers at vtk.org>, "
>>> paraview at paraview.org"
>>> >> >>>> <paraview at paraview.org>
>>> >> >>>> Subject: [EXTERNAL] [Paraview] vtkDistributedDataFilter and ghost
>>> >> >>>> cells
>>> >> >>>>
>>> >> >>>> Hi,
>>> >> >>>>
>>> >> >>>> Hope you can help. I have some code running in parallel, that by
>>> >> >>>> other
>>> >> >>>> means I have constructed nprocs worth of vtkRectilinearGrids,
>>> one per
>>> >> >>>> process. Each of which is a valid nprocs-worth of the whole
>>> serial
>>> >> >>>> mesh,
>>> >> >>>> I've check this and I am happy with that i.e. it's partitioned
>>> >> >>>> properly and
>>> >> >>>> nothing is missing. I need the following information to process
>>> my
>>> >> >>>> data in
>>> >> >>>> parallel:
>>> >> >>>>
>>> >> >>>> 1) I would like the local -> global cell mapping between the
>>> local
>>> >> >>>> rgrid
>>> >> >>>> and the corresponding global single mesh.
>>> >> >>>> 2) I would like to know which cells are on processor boundaries
>>> for
>>> >> >>>> parallel exchange purposes.
>>> >> >>>> 3) I would like all the double arrays per processor to be
>>> "expanded"
>>> >> >>>> by
>>> >> >>>> the amount of (1 level of) ghost cells such that I can properly
>>> do
>>> >> >>>> the
>>> >> >>>> computations I want with the ability to exchange only those
>>> >> >>>> additional cells
>>> >> >>>> given the local to global mapping.
>>> >> >>>>
>>> >> >>>> I have tried from the examples to use the following code, which I
>>> >> >>>> call
>>> >> >>>> on every process, each of which has it's own local rgrid as I
>>> said.
>>> >> >>>> I do
>>> >> >>>> the following:
>>> >> >>>>
>>> >> >>>> vtkSmartPointer<vtkDistributedDataFilter> dd =
>>> >> >>>> vtkSmartPointer<vtkDistributedDataFilter>::New();
>>> >> >>>> dd->SetInput(rgrid);
>>> >> >>>>
>>> >> >>>> dd->SetController(getVtkController());
>>> >> >>>> dd->SetBoundaryModeToSplitBoundaryCells();
>>> >> >>>> //dd->SetBoundaryModeToAssignToOneRegion();
>>> >> >>>> //dd->SetBoundaryModeToAssignToAllIntersectingRegions();
>>> >> >>>> dd->UseMinimalMemoryOff();
>>> >> >>>> dd->Update();
>>> >> >>>> vtkPieceScalars *ps = vtkPieceScalars::New();
>>> >> >>>> ps->SetInputConnection(dd->GetOutputPort());
>>> >> >>>> ps->SetScalarModeToCellData();
>>> >> >>>> vtkDataSetSurfaceFilter *dss = vtkDataSetSurfaceFilter::New();
>>> >> >>>> dss->SetInputConnection(ps->GetOutputPort());
>>> >> >>>>
>>> >> >>>> The dd object works fine and writing its contents out on each
>>> >> >>>> processor
>>> >> >>>> gives nprocs worth of meshes, each of which look slightly
>>> different
>>> >> >>>> to the
>>> >> >>>> way I've partitioned them up, but sum to the same serial mesh so
>>> I am
>>> >> >>>> happy
>>> >> >>>> with that working correctly. But I can't for the life of me
>>> figure
>>> >> >>>> out how
>>> >> >>>> to obtain local to global cell mappings, allocate ghost cells, or
>>> >> >>>> work out
>>> >> >>>> how to exchange data given the above partition info and comms....
>>> >> >>>>
>>> >> >>>> Note I have not provided any additional information to "dd"
>>> regarding
>>> >> >>>> global cells as per the doxy notes so I assume it went away and
>>> >> >>>> computed it.
>>> >> >>>> I can't figure out how to extract it however. I also have no
>>> idea
>>> >> >>>> how to
>>> >> >>>> modify each local processor rgrid to include the ghost cells for
>>> that
>>> >> >>>> processor. Finally given that info, I could exchange between
>>> >> >>>> processors to
>>> >> >>>> write to each local processors ghost cells the corresponding
>>> "real"
>>> >> >>>> cell
>>> >> >>>> data from the neighbouring meshes and continue the code.
>>> >> >>>>
>>> >> >>>> Any help really appreciated!
>>> >> >>>>
>>> >> >>>> Cheers,
>>> >> >>>> Andy
>>> >> >>>>
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> --
>>> >> >>>
>>> >> >>> __________________________________
>>> >> >>>
>>> >> >>> Dr Andrew Parker
>>> >> >>>
>>> >> >>> Em at il: andrew.parker at cantab.net
>>> >> >>>
>>> >> >>>
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> --
>>> >> >>
>>> >> >> __________________________________
>>> >> >>
>>> >> >> Dr Andrew Parker
>>> >> >>
>>> >> >> Em at il: andrew.parker at cantab.net
>>> >> >>
>>> >> >>
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> >
>>> >> > __________________________________
>>> >> >
>>> >> > Dr Andrew Parker
>>> >> >
>>> >> > Em at il: andrew.parker at cantab.net
>>> >> >
>>> >> >
>>> >> >
>>> >> > _______________________________________________
>>> >> > Powered by www.kitware.com
>>> >> >
>>> >> > Visit other Kitware open-source projects at
>>> >> > http://www.kitware.com/opensource/opensource.html
>>> >> >
>>> >> > Please keep messages on-topic and check the VTK FAQ at:
>>> >> > http://www.vtk.org/Wiki/VTK_FAQ
>>> >> >
>>> >> > Follow this link to subscribe/unsubscribe:
>>> >> > http://www.vtk.org/mailman/listinfo/vtkusers
>>> >> >
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> >
>>> > __________________________________
>>> >
>>> > Dr Andrew Parker
>>> >
>>> > Em at il: andrew.parker at cantab.net
>>> >
>>> >
>>> _______________________________________________
>>> Powered by www.kitware.com
>>>
>>> Visit other Kitware open-source projects at
>>> http://www.kitware.com/opensource/opensource.html
>>>
>>> Please keep messages on-topic and check the VTK FAQ at:
>>> http://www.vtk.org/Wiki/VTK_FAQ
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> http://www.vtk.org/mailman/listinfo/vtkusers
>>>
>>
>>
>
>
> --
>
> __________________________________
>
> Dr Andrew Parker
>
> Em at il: andrew.parker at cantab.net
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20121108/77da7e2e/attachment-0001.htm>
More information about the ParaView
mailing list