[Paraview] How to create a parallele reader for paraview ?

[Sebastien Jourdain]

Thanks Kenneth for your feed back.

You have pretty well explained what I meant with the "streaming API".
But to be really sure of how paraview will manage its readers.

Could you correct the following scenario and questions ?

Context:
- We suppose that Paraview is deployed over a cluster of "n" nodes

1) The user want to read a file. (click and select files in the menu)
2) Paraview create "n" instance of the reader. One instance on each 
cluster node.
3) Paraview will apply the same data processing behind each reader by 
setting in vtkInformation of each specific request
    - vtkStreamingDemandDrivenPipeline::UPDATE_PIECE_NUMBER()  equal to 
the MPI_ProcessId
    - and vtkStreamingDemandDrivenPipeline::UPDATE_NUMBER_OF_PIECES() 
equal to the total number of nodes. ("n" for us)
    - and something about GHOST_LEVEL depending on the need of the 
filter involved in the pipeline.

If so, this mean that each reader is reading indepently the same file 
and each one is reading and extracting its requesting part ?
The reader don't need to know anything about MPI. Paraview manage 
everything.
This also mean that the file must be available on the same path on each 
node of the cluster ?
I suppose that the reader will have to provide the requested level of 
ghost cell ?

Which is unfortunately not the behaviour that I'm looking for... ;-)
I will look at the vtkSpyPlotReader and will come back to you if I have 
more question.

Thanks a lot for your support,

Seb


Moreland, Kenneth a écrit :
> Seb,
>
> It sounds like you have a pretty good grasp of how parallel readers work
> in VTK/ParaView.  A reader is expected to respond to the "extent"
> request that comes from the pipeline, which I think is what you mean by
> the "streaming API."  It is used to both specify pieces that are
> streamed in as well as specify pieces per process in a data parallel
> mode (such as used in ParaView).
>
> I cannot think of a reader that works exactly as you describe, but there
> are some that do use a vtkMuliProcessController (vtkSpyPlotReader is
> one).  You can get a controller with
> vtkMultiProcessController::GetGlobalController().  ParaView will
> initialize this global controller with the MPI controller that contains
> MPI_COMM_WORLD.
>
> Does that answer your question?
>
> -Ken
>
>   
>> -----Original Message-----
>> From: paraview-bounces+kmorel=sandia.gov at paraview.org
>>     
> [mailto:paraview-
>   
>> bounces+kmorel=sandia.gov at paraview.org] On Behalf Of Sebastien
>>     
> Jourdain
>   
>> Sent: Thursday, January 17, 2008 2:53 AM
>> To: paraview at paraview.org
>> Subject: [Paraview] How to create a parallele reader for paraview ?
>>
>> Hello,
>>
>> I would like to create a parallele reader that could be used in a
>> standard manner by paraview.
>>
>> My problem, is that I'm not sure how paraview manage it's pipeline
>> distribution. (when it's deployed in MPI over a cluster)
>>
>> I think, (please correct me if I'm wrong)
>> that paraview 3 use the VTK 5 streaming API for the standard reader to
>> split input data among each MPI process by selecting on piece for each
>> MPI process and by simply applying the processing pipeline behind each
>> selected piece.
>> Or if the distribution is not made by the reader, the
>> vtkDistributedDataFilter is used.
>>
>> The thing is, I don't want each reader to read the file. I need only
>>     
> one
>   
>> master reader that will generate n dataset for the n processes. And by
>> now, I don't know how to manage this distribution to allow paraview to
>> add the processing pipeline behind each distributed dataset in a
>> standard manner. I wanted to use the vtkTransmitXXXPiece classes to
>> forward each piece to each process but I can't find any sample on how
>>     
> to
>   
>> make it work with paraview.
>>
>> If someone has exemples or informations I would really apreciate,
>>
>> Thanks,
>>
>> Seb
>>
>> _______________________________________________
>> ParaView mailing list
>> ParaView at paraview.org
>> http://www.paraview.org/mailman/listinfo/paraview
>>     
>
>
>
> .
>
>