[Paraview-developers] [EXTERNAL] Re: Reading multiple files in parallel
Moreland, Kenneth
kmorel at sandia.gov
Tue Aug 16 19:22:30 EDT 2016
The easiest solution is to use vtkAppendFilter to combine the data objects into a single data object.
-Ken
From: Paraview-developers [mailto:paraview-developers-bounces at paraview.org] On Behalf Of sumeet kumar
Sent: Tuesday, August 16, 2016 4:11 PM
To: Moreland, Kenneth <kmorel at sandia.gov>
Cc: ParaView Developers <paraview-developers at paraview.org>; Sumeet Kumar Sinha <skssinha at ucdavis.edu>
Subject: Re: [Paraview-developers] [EXTERNAL] Re: Reading multiple files in parallel
Thanks Kenneth,
I can get the Number_Of_Processes from the UPDATE_NUMBER_OF_PIECES key.
I can read more than one file per request in ParaView on separate VTKDATA_OBJECT (let say array).
I searched on Internet, But, couldn't find how to send an array of VTKDATA_OBJECTS to a single output port.
Currently , I do this
vtkInformation *Node_Mesh = outputVector->GetInformationObject(0);
.....
this->Get_Node_Mesh(UGrid_Node_Mesh);
UGrid_Current_Node_Mesh->ShallowCopy(UGrid_Node_Mesh);
......
vtkUnstructuredGrid *Output_Node_Mesh = vtkUnstructuredGrid::SafeDownCast(Node_Mesh->Get(vtkDataObject::DATA_OBJECT()));
Output_Node_Mesh->ShallowCopy(UGrid_Current_Node_Mesh);
is there any way I can send separate mesh ( VTKDATA_OBJECTS) at once in paraview on the same output port.
Thanks
Sumeet
On Tue, Aug 16, 2016 at 2:28 PM, Moreland, Kenneth <kmorel at sandia.gov<mailto:kmorel at sandia.gov>> wrote:
ParaView will set the UPDATE_NUMBER_OF_PIECES to the number of processes and UPDATE_PIECE_NUMBER to the rank.
-Ken
From: sumeet kumar [mailto:sumeet.kumar507 at gmail.com<mailto:sumeet.kumar507 at gmail.com>]
Sent: Tuesday, August 16, 2016 3:08 PM
To: Moreland, Kenneth <kmorel at sandia.gov<mailto:kmorel at sandia.gov>>; Sumeet Kumar Sinha <skssinha at ucdavis.edu<mailto:skssinha at ucdavis.edu>>
Cc: ParaView Developers <paraview-developers at paraview.org<mailto:paraview-developers at paraview.org>>
Subject: [EXTERNAL] Re: [Paraview-developers] Reading multiple files in parallel
I compiled ParaView with MPI enabled and as you said and now I can get UPDATE_NUMBER_OF_PIECES() and UPDATE_NUMBER_OF_PIECES() keys.
But, as you said in your previous mail that the number of piece requests will not match with the number of files. So, how can I now what is the range of piece requests that will be generated.
For example, if I have 8 files and the user runs para-view on 4 processes. The optimal way is to read two files per processes.
But my question will how can i know that the user is running on 4 nodes only. Is there any key ??/ for that.
Thanks
Sumeet
On Tue, Aug 16, 2016 at 8:03 AM, Moreland, Kenneth <kmorel at sandia.gov<mailto:kmorel at sandia.gov>> wrote:
Are you running ParaView in MPI parallel mode? If not, it might not set the number of pieces.
Also, I don't think you need to set WHOLE_EXTENT. That is for structured type of data.
-Ken
Sent from my iPad so blame autocorrect.
On Aug 15, 2016, at 5:38 PM, sumeet kumar <sumeet.kumar507 at gmail.com<mailto:sumeet.kumar507 at gmail.com>> wrote:
Hey Ken,
I am sorry that I am asking you a lot of stupid questions. But I am stuck here. I followed what you said, But I don't see the piece number getting updated. And also when would that stop being updated...etc. Here is my code
int pvESSI::RequestInformation( vtkInformation *request, vtkInformationVector **vtkNotUsed(inVec), vtkInformationVector* outVec){
this->Initialize();
vtkInformation* Node_Mesh = outVec->GetInformationObject(0);
double Time_range[2]={Time[0],Time[Number_Of_Time_Steps-1]};
Node_Mesh->Set(vtkStreamingDemandDrivenPipeline::TIME_STEPS(),Time, this->Number_Of_Time_Steps);
Node_Mesh->Set(vtkStreamingDemandDrivenPipeline::TIME_RANGE(),Time_range,2);
Node_Mesh->Set(vtkStreamingDemandDrivenPipeline::WHOLE_EXTENT(),EXTENT,6);
int num_of_piec = 10;
Node_Mesh->Set(CAN_HANDLE_PIECE_REQUEST(), 1);
return 1;
}
int pvESSI::RequestData(vtkInformation *vtkNotUsed(request),vtkInformationVector **vtkNotUsed(inputVector), vtkInformationVector *outputVector){
vtkInformation *Node_Mesh = outputVector->GetInformationObject(0);
// outInfo->Print(std::cout);
piece_no = Node_Mesh->Get(vtkStreamingDemandDrivenPipeline::UPDATE_PIECE_NUMBER());
num_of_pieces = Node_Mesh->Get(vtkStreamingDemandDrivenPipeline::UPDATE_NUMBER_OF_PIECES());
cout << "Piece_No " << piece_no << endl;
cout << "Number_of_Pieces " << piece_no << endl;
Step_Initializer(Piece_No); // makes the file and variables ready for that piece
this->Node_Mesh_Current_Time = Time_Map.find( Node_Mesh->Get(vtkStreamingDemandDrivenPipeline::UPDATE_TIME_STEP()))->second;
if (!Whether_Node_Mesh_Build){
this->Get_Node_Mesh(UGrid_Node_Mesh);
UGrid_Current_Node_Mesh->ShallowCopy(UGrid_Node_Mesh);
}
Build_Node_Attributes(UGrid_Current_Node_Mesh, this->Node_Mesh_Current_Time );
Build_Stress_Field_At_Nodes(UGrid_Current_Node_Mesh, this->Node_Mesh_Current_Time);
// get the ouptut pointer to paraview
vtkUnstructuredGrid *Output_Node_Mesh = vtkUnstructuredGrid::SafeDownCast(Node_Mesh->Get(vtkDataObject::DATA_OBJECT()));
Output_Node_Mesh->ShallowCopy(UGrid_Current_Node_Mesh);
return 1;
}
When I run this code, The Piece_no and Number_of_Pieces both remain 0.
Sumeet
On Mon, Aug 15, 2016 at 3:58 PM, Moreland, Kenneth <kmorel at sandia.gov<mailto:kmorel at sandia.gov>> wrote:
During the RequestData pass of the pipeline, you should be able to get the keys vtkStreamingDemandDrivenPipeline::UPDATE_PIECE_NUMBER() and vtkStreamingDemandDrivenPipeline::UPDATE_NUMBER_OF_PIECES() from the output information object that comes from downstream. This will tell you how many pieces the pipeline expects and which piece is expected to be read.
Generally the number of pieces requested will not match exactly the number of files that you have. This means you will have to adjust by either reading more than 1 file at a time or sometimes reading nothing and returning an empty object.
-Ken
From: sumeet kumar [mailto:sumeet.kumar507 at gmail.com<mailto:sumeet.kumar507 at gmail.com>]
Sent: Monday, August 15, 2016 3:52 PM
To: Moreland, Kenneth <kmorel at sandia.gov<mailto:kmorel at sandia.gov>>
Cc: ParaView Developers <paraview-developers at paraview.org<mailto:paraview-developers at paraview.org>>
Subject: [EXTERNAL] Re: [Paraview-developers] Reading multiple files in parallel
Thanks Kenneth, for you help.
I get the vtkAlgorithm::CAN_HANDLE_PIECE_REQUEST(), set 1 part.
But, what to do next.
How can i get the piece no, so that i can update the data for it.
vtkmXMLPUnstructuredGridDataReader is not doing me a great help.
Sumeet
On Mon, Aug 15, 2016 at 12:57 PM, Moreland, Kenneth <kmorel at sandia.gov<mailto:kmorel at sandia.gov>> wrote:
I think the key you are looking for is vtkAlgorithm::CAN_HANDLE_PIECE_REQUEST(), which you want to set to 1 to signify that you can provide multiple pieces. You might want to look at vtkmXMLPUnstructuredGridDataReader for an example.
If I remember correctly, this has recently been changed in VTK. It used to be that you would give the number of pieces you supported, but that functionality was not useful in practice and consequently readers reported that they could produce any number of pieces in order to be useful. Thus, the behavior was change to allow readers simply to report whether or not they support multiple pieces. Berk can probably provide more information.
-Ken
From: Paraview-developers [mailto:paraview-developers-bounces at paraview.org<mailto:paraview-developers-bounces at paraview.org>] On Behalf Of sumeet kumar
Sent: Monday, August 15, 2016 12:58 PM
To: ParaView Developers <paraview-developers at paraview.org<mailto:paraview-developers at paraview.org>>
Subject: [EXTERNAL] [Paraview-developers] Reading multiple files in parallel
Hello all,
I have a number of files which contains information about pieces of unstructured mesh. For example :-
t_2.h5.01.feioutput
t_2.h5.03.feioutput
t_2.h5.04.feioutput
t_2.h5.05.feioutput
t_2.h5.06.feioutput
t_2.h5.07.feioutput
t_2.h5.08.feioutput
t_2.h5.09.feioutput
t_2.h5.feioutput
My reader can read each file individually but I want to read them all in parallel so that I can visualize the whole model. I looked at the http://www.paraview.org/Wiki/Writing_ParaView_Readers and followed the steps for unstructured mesh.
When I try to set the number of pieces in Request Information function as this
vtkInformation* Node_Mesh = outVec->GetInformationObject(0);
Node_Mesh->Set(vtkStreamingDemandDrivenPipeline::MAXIMUM_NUMBER_OF_PIECES(), 10);
I get the following error
error: ‘MAXIMUM_NUMBER_OF_PIECES’ is not a member of ‘vtkStreamingDemandDrivenPipeline’
Node_Mesh->Set(vtkStreamingDemandDrivenPipeline::MAXIMUM_NUMBER_OF_PIECES(), 10);
Also, it would be highly appreciated if anyone could show/(send me link) me to a sample example.
Regards
--
Error! Filename not specified.
Sumeet Kumar Sinha
Graduate Student
Phone: (+1) <tel:%28%2B91%29%209910516219> 5306018271<tel:5306018271>
Website : http://www.sumeetksinha.com/
--
<image001.jpg>
Sumeet Kumar Sinha
Graduate Student
Phone: (+1) <tel:%28%2B91%29%209910516219> 5306018271<tel:5306018271>
Website : http://www.sumeetksinha.com/
--
[Image removed by sender.]
Sumeet Kumar Sinha
Graduate Student
Phone: (+1) <tel:%28%2B91%29%209910516219> 5306018271<tel:5306018271>
Website : http://www.sumeetksinha.com/
--
[Image removed by sender.]
Sumeet Kumar Sinha
Graduate Student
Phone: (+1) <tel:%28%2B91%29%209910516219> 5306018271<tel:5306018271>
Website : http://www.sumeetksinha.com/
--
[http://faculty.engineering.ucdavis.edu/ceetemplate/wp-content/uploads/2013/06/CEE75percent500w.jpg]
Sumeet Kumar Sinha
Graduate Student
Phone: (+1) <tel:%28%2B91%29%209910516219> 5306018271
Website : http://www.sumeetksinha.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview-developers/attachments/20160816/29d16e6e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 407 bytes
Desc: image001.jpg
URL: <http://public.kitware.com/pipermail/paraview-developers/attachments/20160816/29d16e6e/attachment-0001.jpg>
More information about the Paraview-developers
mailing list