[Paraview-developers] Reading multiple files in parallel
sumeet kumar
sumeet.kumar507 at gmail.com
Tue Aug 16 17:07:57 EDT 2016
I compiled ParaView with MPI enabled and as you said and now I can get
UPDATE_NUMBER_OF_PIECES() and UPDATE_NUMBER_OF_PIECES() keys.
But, as you said in your previous mail that the number of piece requests
will not match with the number of files. So, how can I now what is the
range of piece requests that will be generated.
For example, if I have 8 files and the user runs para-view on 4 processes.
The optimal way is to read two files per processes.
But my question will how can i know that the user is running on 4 nodes
only. Is there any key ??/ for that.
Thanks
Sumeet
On Tue, Aug 16, 2016 at 8:03 AM, Moreland, Kenneth <kmorel at sandia.gov>
wrote:
> Are you running ParaView in MPI parallel mode? If not, it might not set
> the number of pieces.
>
> Also, I don't think you need to set WHOLE_EXTENT. That is for structured
> type of data.
>
> -Ken
>
> Sent from my iPad so blame autocorrect.
>
> On Aug 15, 2016, at 5:38 PM, sumeet kumar <sumeet.kumar507 at gmail.com>
> wrote:
>
> Hey Ken,
>
> I am sorry that I am asking you a lot of stupid questions. But I am stuck
> here. I followed what you said, But I don't see the piece number getting
> updated. And also when would that stop being updated...etc. Here is my code
>
> int pvESSI::RequestInformation( vtkInformation *request,
> vtkInformationVector **vtkNotUsed(inVec), vtkInformationVector* outVec){
>
> this->Initialize();
>
> vtkInformation* Node_Mesh = outVec->GetInformationObject(0);
>
> double Time_range[2]={Time[0],Time[Number_Of_Time_Steps-1]};
>
> Node_Mesh->Set(vtkStreamingDemandDrivenPipeline::TIME_STEPS(),Time,
> this->Number_Of_Time_Steps);
> Node_Mesh->Set(vtkStreamingDemandDrivenPipeli
> ne::TIME_RANGE(),Time_range,2);
>
> Node_Mesh->Set(vtkStreamingDemandDrivenPipeli
> ne::WHOLE_EXTENT(),EXTENT,6);
>
> int num_of_piec = 10;
> Node_Mesh->Set(CAN_HANDLE_PIECE_REQUEST(), 1);
>
> return 1;
> }
>
> int pvESSI::RequestData(vtkInformation *vtkNotUsed(request),vtkInformationVector
> **vtkNotUsed(inputVector), vtkInformationVector *outputVector){
>
> vtkInformation *Node_Mesh = outputVector->GetInformationObject(0);
> // outInfo->Print(std::cout);
>
> piece_no = Node_Mesh->Get(vtkStreamingDemandDrivenPipeli
> ne::UPDATE_PIECE_NUMBER());
> num_of_pieces = Node_Mesh->Get(vtkStreamingDemandDrivenPipeli
> ne::UPDATE_NUMBER_OF_PIECES());
> cout << "Piece_No " << piece_no << endl;
> cout << "Number_of_Pieces " << piece_no << endl;
>
> Step_Initializer(Piece_No); // makes the file and variables
> ready for that piece
>
> this->Node_Mesh_Current_Time = Time_Map.find( Node_Mesh->Get(
> vtkStreamingDemandDrivenPipeline::UPDATE_TIME_STEP()))->second;
>
> if (!Whether_Node_Mesh_Build){
> this->Get_Node_Mesh(UGrid_Node_Mesh);
> UGrid_Current_Node_Mesh->ShallowCopy(UGrid_Node_Mesh);
> }
>
> Build_Node_Attributes(UGrid_Current_Node_Mesh,
> this->Node_Mesh_Current_Time );
> Build_Stress_Field_At_Nodes(UGrid_Current_Node_Mesh,
> this->Node_Mesh_Current_Time);
>
> // get the ouptut pointer to paraview
> vtkUnstructuredGrid *Output_Node_Mesh = vtkUnstructuredGrid::
> SafeDownCast(Node_Mesh->Get(vtkDataObject::DATA_OBJECT()));
>
> Output_Node_Mesh->ShallowCopy(UGrid_Current_Node_Mesh);
> return 1;
> }
>
> When I run this code, The * Piece_no* and *Number_of_Pieces* both remain
> 0.
>
>
> Sumeet
>
> On Mon, Aug 15, 2016 at 3:58 PM, Moreland, Kenneth <kmorel at sandia.gov>
> wrote:
>
>> During the RequestData pass of the pipeline, you should be able to get
>> the keys vtkStreamingDemandDrivenPipeline::UPDATE_PIECE_NUMBER() and
>> vtkStreamingDemandDrivenPipeline::UPDATE_NUMBER_OF_PIECES() from the
>> output information object that comes from downstream. This will tell you
>> how many pieces the pipeline expects and which piece is expected to be read.
>>
>>
>>
>> Generally the number of pieces requested will not match exactly the
>> number of files that you have. This means you will have to adjust by either
>> reading more than 1 file at a time or sometimes reading nothing and
>> returning an empty object.
>>
>>
>>
>> -Ken
>>
>>
>>
>> *From:* sumeet kumar [mailto:sumeet.kumar507 at gmail.com]
>> *Sent:* Monday, August 15, 2016 3:52 PM
>> *To:* Moreland, Kenneth <kmorel at sandia.gov>
>> *Cc:* ParaView Developers <paraview-developers at paraview.org>
>> *Subject:* [EXTERNAL] Re: [Paraview-developers] Reading multiple files
>> in parallel
>>
>>
>>
>> Thanks Kenneth, for you help.
>>
>> I get the vtkAlgorithm::CAN_HANDLE_PIECE_REQUEST(), set 1 part.
>>
>> But, what to do next.
>>
>> How can i get the piece no, so that i can update the data for it.
>>
>> vtkmXMLPUnstructuredGridDataReader is not doing me a great help.
>>
>> Sumeet
>>
>>
>>
>> On Mon, Aug 15, 2016 at 12:57 PM, Moreland, Kenneth <kmorel at sandia.gov>
>> wrote:
>>
>> I think the key you are looking for is vtkAlgorithm::CAN_HANDLE_PIECE_REQUEST(),
>> which you want to set to 1 to signify that you can provide multiple pieces.
>> You might want to look at vtkmXMLPUnstructuredGridDataReader for an
>> example.
>>
>>
>>
>> If I remember correctly, this has recently been changed in VTK. It used
>> to be that you would give the number of pieces you supported, but that
>> functionality was not useful in practice and consequently readers reported
>> that they could produce any number of pieces in order to be useful. Thus,
>> the behavior was change to allow readers simply to report whether or not
>> they support multiple pieces. Berk can probably provide more information.
>>
>>
>>
>> -Ken
>>
>>
>>
>> *From:* Paraview-developers [mailto:paraview-developers-bo
>> unces at paraview.org] *On Behalf Of *sumeet kumar
>> *Sent:* Monday, August 15, 2016 12:58 PM
>> *To:* ParaView Developers <paraview-developers at paraview.org>
>> *Subject:* [EXTERNAL] [Paraview-developers] Reading multiple files in
>> parallel
>>
>>
>>
>> Hello all,
>>
>> I have a number of files which contains information about pieces of
>> unstructured mesh. For example :-
>>
>> t_2.h5.01.feioutput
>> t_2.h5.03.feioutput
>> t_2.h5.04.feioutput
>> t_2.h5.05.feioutput
>> t_2.h5.06.feioutput
>> t_2.h5.07.feioutput
>> t_2.h5.08.feioutput
>> t_2.h5.09.feioutput
>> t_2.h5.feioutput
>>
>> My reader can read each file individually but I want to read them all in
>> parallel so that I can visualize the whole model. I looked at the
>> http://www.paraview.org/Wiki/Writing_ParaView_Readers and followed the
>> steps for unstructured mesh.
>>
>> When I try to set the number of pieces in Request Information function as
>> this
>>
>> vtkInformation* Node_Mesh = outVec->GetInformationObject(0);
>> Node_Mesh->Set(vtkStreamingDemandDrivenPipeline::MAXIMUM_NUMBER_OF_PIECES(),
>> 10);
>>
>>
>>
>> I get the following error
>>
>> error: ‘MAXIMUM_NUMBER_OF_PIECES’ is not a member of
>> ‘vtkStreamingDemandDrivenPipeline’
>> Node_Mesh->Set(vtkStreamingDemandDrivenPipeline::MAXIMUM_NUMBER_OF_PIECES(),
>> 10);
>>
>>
>>
>> Also, it would be highly appreciated if anyone could show/(send me link)
>> me to a sample example.
>>
>> Regards
>>
>> --
>>
>> *Error! Filename not specified.*
>>
>> Sumeet Kumar Sinha
>>
>> Graduate Student
>>
>> Phone: (+1) <%28%2B91%29%209910516219>5306018271
>>
>> Website : http://www.sumeetksinha.com/
>>
>>
>>
>>
>> --
>>
>> <image001.jpg>
>>
>> Sumeet Kumar Sinha
>>
>> Graduate Student
>>
>> Phone: (+1) <%28%2B91%29%209910516219>5306018271
>>
>> Website : http://www.sumeetksinha.com/
>>
>
>
>
> --
>
> Sumeet Kumar Sinha
> Graduate Student
> Phone: (+1) <%28%2B91%29%209910516219>5306018271
> Website : http://www.sumeetksinha.com/
>
>
--
Sumeet Kumar Sinha
Graduate Student
Phone: (+1) <%28%2B91%29%209910516219>5306018271
Website : http://www.sumeetksinha.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview-developers/attachments/20160816/8c104ee8/attachment-0001.html>
More information about the Paraview-developers
mailing list