[Paraview-developers] [EXTERNAL] Re: Reading multiple files in parallel
sumeet kumar
sumeet.kumar507 at gmail.com
Wed Aug 17 13:02:44 EDT 2016
Thank You a lot Kenneth,
It worked finally :)
Regards
Sumeet
On Tue, Aug 16, 2016 at 4:22 PM, Moreland, Kenneth <kmorel at sandia.gov>
wrote:
> The easiest solution is to use vtkAppendFilter to combine the data objects
> into a single data object.
>
>
>
> -Ken
>
>
>
> *From:* Paraview-developers [mailto:paraview-developers-
> bounces at paraview.org] *On Behalf Of *sumeet kumar
> *Sent:* Tuesday, August 16, 2016 4:11 PM
> *To:* Moreland, Kenneth <kmorel at sandia.gov>
> *Cc:* ParaView Developers <paraview-developers at paraview.org>; Sumeet
> Kumar Sinha <skssinha at ucdavis.edu>
> *Subject:* Re: [Paraview-developers] [EXTERNAL] Re: Reading multiple
> files in parallel
>
>
>
> Thanks Kenneth,
>
> I can get the Number_Of_Processes from the UPDATE_NUMBER_OF_PIECES key.
>
> I can read more than one file per request in ParaView on separate
> VTKDATA_OBJECT (let say array).
>
> I searched on Internet, But, couldn't find how to send an array of
> VTKDATA_OBJECTS to a single output port.
>
> Currently , I do this
>
> vtkInformation *Node_Mesh = outputVector->GetInformationObject(0);
>
> .....
>
>
> this->Get_Node_Mesh(UGrid_Node_Mesh);
> UGrid_Current_Node_Mesh->ShallowCopy(UGrid_Node_Mesh);
>
>
> ......
>
>
> vtkUnstructuredGrid *Output_Node_Mesh = vtkUnstructuredGrid::
> SafeDownCast(Node_Mesh->Get(vtkDataObject::DATA_OBJECT()));
>
> Output_Node_Mesh->ShallowCopy(UGrid_Current_Node_Mesh);
>
> is there any way I can send separate mesh ( VTKDATA_OBJECTS) at once in
> paraview on the same output port.
>
> Thanks
>
> Sumeet
>
>
>
>
>
> On Tue, Aug 16, 2016 at 2:28 PM, Moreland, Kenneth <kmorel at sandia.gov>
> wrote:
>
> ParaView will set the UPDATE_NUMBER_OF_PIECES to the number of processes
> and UPDATE_PIECE_NUMBER to the rank.
>
>
>
> -Ken
>
>
>
> *From:* sumeet kumar [mailto:sumeet.kumar507 at gmail.com]
> *Sent:* Tuesday, August 16, 2016 3:08 PM
> *To:* Moreland, Kenneth <kmorel at sandia.gov>; Sumeet Kumar Sinha <
> skssinha at ucdavis.edu>
>
>
> *Cc:* ParaView Developers <paraview-developers at paraview.org>
> *Subject:* [EXTERNAL] Re: [Paraview-developers] Reading multiple files in
> parallel
>
>
>
> I compiled ParaView with MPI enabled and as you said and now I can get
> UPDATE_NUMBER_OF_PIECES() and UPDATE_NUMBER_OF_PIECES() keys.
>
> But, as you said in your previous mail that the number of piece requests
> will not match with the number of files. So, how can I now what is the
> range of piece requests that will be generated.
>
> For example, if I have 8 files and the user runs para-view on 4
> processes. The optimal way is to read two files per processes.
>
> But my question will how can i know that the user is running on 4 nodes
> only. Is there any key ??/ for that.
>
> Thanks
>
> Sumeet
>
>
>
>
>
> On Tue, Aug 16, 2016 at 8:03 AM, Moreland, Kenneth <kmorel at sandia.gov>
> wrote:
>
> Are you running ParaView in MPI parallel mode? If not, it might not set
> the number of pieces.
>
>
>
> Also, I don't think you need to set WHOLE_EXTENT. That is for structured
> type of data.
>
>
>
> -Ken
>
>
>
> Sent from my iPad so blame autocorrect.
>
>
> On Aug 15, 2016, at 5:38 PM, sumeet kumar <sumeet.kumar507 at gmail.com>
> wrote:
>
> Hey Ken,
>
> I am sorry that I am asking you a lot of stupid questions. But I am stuck
> here. I followed what you said, But I don't see the piece number getting
> updated. And also when would that stop being updated...etc. Here is my code
>
> int pvESSI::RequestInformation( vtkInformation *request,
> vtkInformationVector **vtkNotUsed(inVec), vtkInformationVector* outVec){
>
> this->Initialize();
>
> vtkInformation* Node_Mesh = outVec->GetInformationObject(0);
>
> double Time_range[2]={Time[0],Time[Number_Of_Time_Steps-1]};
>
> Node_Mesh->Set(vtkStreamingDemandDrivenPipeline::TIME_STEPS(),Time,
> this->Number_Of_Time_Steps);
> Node_Mesh->Set(vtkStreamingDemandDrivenPipeli
> ne::TIME_RANGE(),Time_range,2);
>
> Node_Mesh->Set(vtkStreamingDemandDrivenPipeli
> ne::WHOLE_EXTENT(),EXTENT,6);
>
> int num_of_piec = 10;
> Node_Mesh->Set(CAN_HANDLE_PIECE_REQUEST(), 1);
>
> return 1;
> }
>
> int pvESSI::RequestData(vtkInformation *vtkNotUsed(request),vtkInformationVector
> **vtkNotUsed(inputVector), vtkInformationVector *outputVector){
>
> vtkInformation *Node_Mesh = outputVector->GetInformationObject(0);
> // outInfo->Print(std::cout);
>
> piece_no = Node_Mesh->Get(vtkStreamingDemandDrivenPipeli
> ne::UPDATE_PIECE_NUMBER());
> num_of_pieces = Node_Mesh->Get(vtkStreamingDemandDrivenPipeli
> ne::UPDATE_NUMBER_OF_PIECES());
> cout << "Piece_No " << piece_no << endl;
> cout << "Number_of_Pieces " << piece_no << endl;
>
> Step_Initializer(Piece_No); // makes the file and variables
> ready for that piece
>
> this->Node_Mesh_Current_Time = Time_Map.find( Node_Mesh->Get(
> vtkStreamingDemandDrivenPipeline::UPDATE_TIME_STEP()))->second;
>
> if (!Whether_Node_Mesh_Build){
> this->Get_Node_Mesh(UGrid_Node_Mesh);
> UGrid_Current_Node_Mesh->ShallowCopy(UGrid_Node_Mesh);
> }
>
> Build_Node_Attributes(UGrid_Current_Node_Mesh,
> this->Node_Mesh_Current_Time );
> Build_Stress_Field_At_Nodes(UGrid_Current_Node_Mesh,
> this->Node_Mesh_Current_Time);
>
> // get the ouptut pointer to paraview
> vtkUnstructuredGrid *Output_Node_Mesh = vtkUnstructuredGrid::
> SafeDownCast(Node_Mesh->Get(vtkDataObject::DATA_OBJECT()));
>
> Output_Node_Mesh->ShallowCopy(UGrid_Current_Node_Mesh);
> return 1;
> }
>
> When I run this code, The *Piece_no* and *Number_of_Pieces* both remain 0.
>
> Sumeet
>
>
>
> On Mon, Aug 15, 2016 at 3:58 PM, Moreland, Kenneth <kmorel at sandia.gov>
> wrote:
>
> During the RequestData pass of the pipeline, you should be able to get the
> keys vtkStreamingDemandDrivenPipeline::UPDATE_PIECE_NUMBER() and
> vtkStreamingDemandDrivenPipeline::UPDATE_NUMBER_OF_PIECES() from the
> output information object that comes from downstream. This will tell you
> how many pieces the pipeline expects and which piece is expected to be read.
>
>
>
> Generally the number of pieces requested will not match exactly the number
> of files that you have. This means you will have to adjust by either
> reading more than 1 file at a time or sometimes reading nothing and
> returning an empty object.
>
>
>
> -Ken
>
>
>
> *From:* sumeet kumar [mailto:sumeet.kumar507 at gmail.com]
> *Sent:* Monday, August 15, 2016 3:52 PM
> *To:* Moreland, Kenneth <kmorel at sandia.gov>
> *Cc:* ParaView Developers <paraview-developers at paraview.org>
> *Subject:* [EXTERNAL] Re: [Paraview-developers] Reading multiple files in
> parallel
>
>
>
> Thanks Kenneth, for you help.
>
> I get the vtkAlgorithm::CAN_HANDLE_PIECE_REQUEST(), set 1 part.
>
> But, what to do next.
>
> How can i get the piece no, so that i can update the data for it.
>
> vtkmXMLPUnstructuredGridDataReader is not doing me a great help.
>
> Sumeet
>
>
>
> On Mon, Aug 15, 2016 at 12:57 PM, Moreland, Kenneth <kmorel at sandia.gov>
> wrote:
>
> I think the key you are looking for is vtkAlgorithm::CAN_HANDLE_PIECE_REQUEST(),
> which you want to set to 1 to signify that you can provide multiple pieces.
> You might want to look at vtkmXMLPUnstructuredGridDataReader for an
> example.
>
>
>
> If I remember correctly, this has recently been changed in VTK. It used to
> be that you would give the number of pieces you supported, but that
> functionality was not useful in practice and consequently readers reported
> that they could produce any number of pieces in order to be useful. Thus,
> the behavior was change to allow readers simply to report whether or not
> they support multiple pieces. Berk can probably provide more information.
>
>
>
> -Ken
>
>
>
> *From:* Paraview-developers [mailto:paraview-developers-
> bounces at paraview.org] *On Behalf Of *sumeet kumar
> *Sent:* Monday, August 15, 2016 12:58 PM
> *To:* ParaView Developers <paraview-developers at paraview.org>
> *Subject:* [EXTERNAL] [Paraview-developers] Reading multiple files in
> parallel
>
>
>
> Hello all,
>
> I have a number of files which contains information about pieces of
> unstructured mesh. For example :-
>
> t_2.h5.01.feioutput
> t_2.h5.03.feioutput
> t_2.h5.04.feioutput
> t_2.h5.05.feioutput
> t_2.h5.06.feioutput
> t_2.h5.07.feioutput
> t_2.h5.08.feioutput
> t_2.h5.09.feioutput
> t_2.h5.feioutput
>
> My reader can read each file individually but I want to read them all in
> parallel so that I can visualize the whole model. I looked at the
> http://www.paraview.org/Wiki/Writing_ParaView_Readers and followed the
> steps for unstructured mesh.
>
> When I try to set the number of pieces in Request Information function as
> this
>
> vtkInformation* Node_Mesh = outVec->GetInformationObject(0);
> Node_Mesh->Set(vtkStreamingDemandDrivenPipeli
> ne::MAXIMUM_NUMBER_OF_PIECES(), 10);
>
>
>
> I get the following error
>
> error: ‘MAXIMUM_NUMBER_OF_PIECES’ is not a member of ‘
> vtkStreamingDemandDrivenPipeline’
> Node_Mesh->Set(vtkStreamingDemandDrivenPipeli
> ne::MAXIMUM_NUMBER_OF_PIECES(), 10);
>
>
>
> Also, it would be highly appreciated if anyone could show/(send me link)
> me to a sample example.
>
> Regards
>
> --
>
> *Error! Filename not specified.*
>
> Sumeet Kumar Sinha
>
> Graduate Student
>
> Phone: (+1) <%28%2B91%29%209910516219>5306018271
>
> Website : http://www.sumeetksinha.com/
>
>
>
>
> --
>
> <image001.jpg>
>
> Sumeet Kumar Sinha
>
> Graduate Student
>
> Phone: (+1) <%28%2B91%29%209910516219>5306018271
>
> Website : http://www.sumeetksinha.com/
>
>
>
>
> --
>
> [image: Image removed by sender.]
>
> Sumeet Kumar Sinha
>
> Graduate Student
>
> Phone: (+1) <%28%2B91%29%209910516219>5306018271
>
> Website : http://www.sumeetksinha.com/
>
>
>
>
> --
>
> [image: Image removed by sender.]
>
> Sumeet Kumar Sinha
>
> Graduate Student
>
> Phone: (+1) <%28%2B91%29%209910516219>5306018271
>
> Website : http://www.sumeetksinha.com/
>
>
>
>
> --
>
>
>
> Sumeet Kumar Sinha
>
> Graduate Student
>
> Phone: (+1) <%28%2B91%29%209910516219>5306018271
>
> Website : http://www.sumeetksinha.com/
>
--
Sumeet Kumar Sinha
Graduate Student
Phone: (+1) <%28%2B91%29%209910516219>5306018271
Website : http://www.sumeetksinha.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview-developers/attachments/20160817/5333085c/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 407 bytes
Desc: not available
URL: <http://public.kitware.com/pipermail/paraview-developers/attachments/20160817/5333085c/attachment-0001.jpg>
More information about the Paraview-developers
mailing list