[Paraview] parallel filter question

[Moreland, Kenneth]

Mark,

That is more clear. I suspect the crash you are seeing is actually the opposite problem. Although you are correct in that there is not a magic scatter/gather in between every filter, the integrate attributes filter is special in that it is parallel aware. At the end of its operation it does a reduce internal to the operation of the filter. It returns valid data in node 0 and an empty data set in all other nodes. I bet your crash occurs when you try to use one of these empty data sets.

-Ken

From: <Vanmoer>, Mark W <mvanmoer at illinois.edu<mailto:mvanmoer at illinois.edu>>
Date: Friday, March 28, 2014 1:03 PM
To: Kenneth Moreland <kmorel at sandia.gov<mailto:kmorel at sandia.gov>>, David E DeMarle <dave.demarle at kitware.com<mailto:dave.demarle at kitware.com>>
Cc: paraview <paraview at paraview.org<mailto:paraview at paraview.org>>
Subject: [EXTERNAL] RE: [Paraview] parallel filter question

Ken and David,

The end goal is to be able to chart the ratio of different volumes over a time series.

I started with a hand built pipeline in the client: a wavelet source and a threshold. Attached to each is an Integrate Variables filter which provides the volumes and the ratio is calculated with a Python calculator filter. Then I created a custom filter out of this pipeline. However I discovered that the custom filter is particular about how it is setup. E.g., since it was setup with a wavelet source, it expects a wavelet source and if I attach it to a vtk image data set the result is garbage.

That was when I figured I could write a VTK filter and load that as a plugin. I wrote a multiple input connection filter that creates two vtkIntegrateAttribute ivars and calculates the ratio. Following vtkIntegrateAttributes, it outputs the result as a scalar attached to a vertex cell. This works great in serial. Crashes as soon as I attach it to a pipeline when I’m running in parallel.

For some reason I had it in my head that each filter stage of the pipeline did a scatter and gather and that my ivars would magically be gathering for me. But when I was reviewing the ParaView guide it finally sunk in that I was getting 2 vtkIntegrateAttribute ivars for each process and that the only gathering was the final image compositing.

Mark

From: Moreland, Kenneth [mailto:kmorel at sandia.gov]
Sent: Friday, March 28, 2014 10:22 AM
To: David E DeMarle; Vanmoer, Mark W
Cc: paraview
Subject: Re: [Paraview] parallel filter question

Mark,

By the way, we might be able to answer your question better if you gave extra information about what you are trying to do. Short of writing a book on the ins and outs of parallel processing in ParaView, we can only guess what you might need.

Most filters, even though they are run in parallel only need to consider the local partition of data they are given. There are, of course, exceptions, but if you give us some hints at what you are trying to do we might be able to give you better advice.

-Ken


From: David E DeMarle <dave.demarle at kitware.com<mailto:dave.demarle at kitware.com>>
Date: Thursday, March 27, 2014 9:21 AM
To: "Vanmoer, Mark W" <mvanmoer at illinois.edu<mailto:mvanmoer at illinois.edu>>
Cc: paraview <paraview at paraview.org<mailto:paraview at paraview.org>>
Subject: [EXTERNAL] Re: [Paraview] parallel filter question

Pretty much yeah. ParaView does interprocess communication as little as it can.

Grep for vtkMPIController in ParaViewCore/VTKExtensions and VTK/Filters/Parallel for examples of the mechanics of having parallel filters communicate amongst themselves.

David E DeMarle
Kitware, Inc.
R&D Engineer
21 Corporate Drive
Clifton Park, NY 12065-8662
Phone: 518-881-4909

On Thu, Mar 27, 2014 at 11:00 AM, Vanmoer, Mark W <mvanmoer at illinois.edu<mailto:mvanmoer at illinois.edu>> wrote:
Hi all, I’m actually new to nuts and bolts parallel programming, so this question might be a bit basic. I was re-reading section 13.4 of the ParaView Guide and wanted to clarify, is it correct to say that ParaView only does gathering for the image output that IceT is going to composite? That for filters that need to use any other data from the pipeline for non-image output, that gathering has to be done by hand?

Mark

_______________________________________________
Powered by www.kitware.com<http://www.kitware.com>

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView

Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20140328/bab4b134/attachment-0001.html>