[Paraview] Parallel Python Programmable Filter Requesting Additional Ghost Data

[Jesus Pulido]

I am simply applying a (Python) Programmable Filter after loading in the
data so the data itself does not get touched or modified before the
Programmable Filter is applied. Rebuilding paraview from git won't be a
problem since I've branched a few git versions in the past. If possible,
any workaround (including the previously suggested) would be great since
I'd rather not code my own MPI ghost cell trading logic inside of my
parallel programmable filter as I'm already performing computations every
single time I run the filter. I'd imagine it would be much slower than
having the ghost data already present or generating such ghost data
beforehand.

On Tue, Jul 17, 2012 at 2:05 PM, David E DeMarle
<dave.demarle at kitware.com>wrote:

> Also remember this gem, which may help you.
>
> >Note also that there is a "feature" in the python programmable filter
> >that comes into play with structured data. That feature says that
> >structured data is not split at all by default. If you want structured
> >data to actually be parallel you need to put this code in your python
> >programmable filter.
> >
> >from paraview import util
> >self.GetExecutive().
>>
>> SetExtentTranslator(self.GetExecutive().GetOutputInformation(0),
>> >vtk.vtkExtentTranslator())
>
>
Yes, I currently use that command to split my data for my Programmable
Filter. I ran into this "feature" last year.

Thank you,
Jesus

On Tue, Jul 17, 2012 at 2:02 PM, Berk Geveci <berk.geveci at kitware.com>wrote:

> One more question. What are you doing after reading the data? Depending on
> what you are doing, I might be able to give you a piece of code that uses
> George's filter to generate ghost levels using MPI. You will need to use
> ParaView built from git.
>
> Next week, George and I will start working on updating ParaView's
> structured data readers to generate ghost levels using MPI under covers. We
> waited way too long to do it...
>
> -berk
>
>
> On Tue, Jul 17, 2012 at 3:34 PM, Jesus Pulido <jpulido at ucdavis.edu> wrote:
>
>> Yes, appended raw mode. It has an ascii header then the data itself is
>> binary after a specified underscore delimiter.
>>
>> Jesus
>>
>>
>> On Tue, Jul 17, 2012 at 1:17 PM, Berk Geveci <berk.geveci at kitware.com>wrote:
>>
>>> Actually, I think that I noticed this too. It may be a bug specific to
>>> the programmable filter. I will check it out. By the way, what do you mean
>>> by "raw" vti? Appended raw mode?
>>>
>>> -berk
>>>
>>>
>>> On Tue, Jul 17, 2012 at 12:22 PM, Jesus Pulido <jpulido at ucdavis.edu>wrote:
>>>
>>>> Thank you Berk.
>>>>
>>>> I was under the impression that this command would trigger the reader
>>>> to automatically load in the additional # of requested ghost levels. I
>>>> tested the command and each data piece still has the same extents as before.
>>>>
>>>> Is there any additional way to load in ghost level that's parallel
>>>> friendly? I am aware of the D3 filter but, from the memory usage, it seems
>>>> that it will load in the entire dataset into each processor and this is not
>>>> feasible for large datasets.
>>>>
>>>> I am using binary, "raw" vti piece files each with their own extent of
>>>> the data and a header pvti file.
>>>> The data has no encoded ghost layers between pieces so I was hoping to
>>>> have paraview do that for me.
>>>>
>>>> Jesus
>>>>
>>>>
>>>> On Tue, Jul 17, 2012 at 7:29 AM, Berk Geveci <berk.geveci at kitware.com>wrote:
>>>>
>>>>> self.GetExecutive().GetOutputInformation(0).Set(vtk.vtkStreamingDemandDrivenPipeline.UPDATE_NUMBER_OF_GHOST_LEVELS(),
>>>>> 1)
>>>>>
>>>>>
>>>>> in RequestUpdateExtent script. Of course, this requires that the
>>>>> reader or a filter before the programmable filter is able to provide ghost
>>>>> levels.
>>>>>
>>>>>
>>>>> -berk
>>>>>
>>>>>  On Wed, Jul 11, 2012 at 3:04 PM, Jesus Pulido <jpulido at ucdavis.edu>wrote:
>>>>>
>>>>>>  Hello,
>>>>>>
>>>>>> I am trying to request ghost data in my python programmable filter
>>>>>> but I am not sure how to go about and do so. I am assuming the call would
>>>>>> go into the RequestUpdateExtent Script.
>>>>>>
>>>>>> In the C++ equivalent filters that do request ghost data, the call to
>>>>>> is something along the lines of
>>>>>> vtkStreamingDemandDrivenPipeline::UPDATE_NUMBER_OF_GHOST_LEVELS() ,
>>>>>> and there are examples of that in the Paraview source but the way the
>>>>>> objects are defined and called does not translate into Paraview VTK Python.
>>>>>> What would be the appropriate call?
>>>>>>
>>>>>> Thank you,
>>>>>> Jesus
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Powered by www.kitware.com
>>>>>>
>>>>>> Visit other Kitware open-source projects at
>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>
>>>>>> Please keep messages on-topic and check the ParaView Wiki at:
>>>>>> http://paraview.org/Wiki/ParaView
>>>>>>
>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>> http://www.paraview.org/mailman/listinfo/paraview
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20120717/cda17320/attachment.htm>