[Paraview-developers] Reading multiple files in parallel

Tue Aug 16 11:03:50 EDT 2016

Are you running ParaView in MPI parallel mode? If not, it might not set the number of pieces.

Also, I don't think you need to set WHOLE_EXTENT. That is for structured type of data.

-Ken

Sent from my iPad so blame autocorrect.

On Aug 15, 2016, at 5:38 PM, sumeet kumar <sumeet.kumar507 at gmail.com<mailto:sumeet.kumar507 at gmail.com>> wrote:

Hey Ken,

I am sorry that I am asking you a lot of stupid questions. But I am stuck here. I followed what you said, But I don't see the piece number getting updated. And also when would that stop being updated...etc. Here is my code

int pvESSI::RequestInformation( vtkInformation *request, vtkInformationVector **vtkNotUsed(inVec), vtkInformationVector* outVec){

    this->Initialize();

    vtkInformation* Node_Mesh = outVec->GetInformationObject(0);

    double Time_range[2]={Time[0],Time[Number_Of_Time_Steps-1]};

    Node_Mesh->Set(vtkStreamingDemandDrivenPipeline::TIME_STEPS(),Time, this->Number_Of_Time_Steps);
    Node_Mesh->Set(vtkStreamingDemandDrivenPipeline::TIME_RANGE(),Time_range,2);

    Node_Mesh->Set(vtkStreamingDemandDrivenPipeline::WHOLE_EXTENT(),EXTENT,6);

    int num_of_piec = 10;
    Node_Mesh->Set(CAN_HANDLE_PIECE_REQUEST(), 1);

    return 1;
}

int pvESSI::RequestData(vtkInformation *vtkNotUsed(request),vtkInformationVector **vtkNotUsed(inputVector),    vtkInformationVector *outputVector){

     vtkInformation *Node_Mesh = outputVector->GetInformationObject(0);
    // outInfo->Print(std::cout);

    piece_no = Node_Mesh->Get(vtkStreamingDemandDrivenPipeline::UPDATE_PIECE_NUMBER());
    num_of_pieces = Node_Mesh->Get(vtkStreamingDemandDrivenPipeline::UPDATE_NUMBER_OF_PIECES());
    cout << "Piece_No " << piece_no << endl;
    cout << "Number_of_Pieces " << piece_no << endl;

     Step_Initializer(Piece_No);     // makes the file and variables ready for that piece

      this->Node_Mesh_Current_Time = Time_Map.find( Node_Mesh->Get(vtkStreamingDemandDrivenPipeline::UPDATE_TIME_STEP()))->second;

    if (!Whether_Node_Mesh_Build){
        this->Get_Node_Mesh(UGrid_Node_Mesh);
        UGrid_Current_Node_Mesh->ShallowCopy(UGrid_Node_Mesh);
    }

    Build_Node_Attributes(UGrid_Current_Node_Mesh, this->Node_Mesh_Current_Time );
    Build_Stress_Field_At_Nodes(UGrid_Current_Node_Mesh, this->Node_Mesh_Current_Time);

    // get the ouptut pointer to paraview
    vtkUnstructuredGrid *Output_Node_Mesh = vtkUnstructuredGrid::SafeDownCast(Node_Mesh->Get(vtkDataObject::DATA_OBJECT()));

    Output_Node_Mesh->ShallowCopy(UGrid_Current_Node_Mesh);
    return 1;
}

When I run this code, The Piece_no and Number_of_Pieces both remain 0.

Sumeet

On Mon, Aug 15, 2016 at 3:58 PM, Moreland, Kenneth <kmorel at sandia.gov<mailto:kmorel at sandia.gov>> wrote:
During the RequestData pass of the pipeline, you should be able to get the keys vtkStreamingDemandDrivenPipeline::UPDATE_PIECE_NUMBER() and vtkStreamingDemandDrivenPipeline::UPDATE_NUMBER_OF_PIECES() from the output information object that comes from downstream. This will tell you how many pieces the pipeline expects and which piece is expected to be read.

Generally the number of pieces requested will not match exactly the number of files that you have. This means you will have to adjust by either reading more than 1 file at a time or sometimes reading nothing and returning an empty object.

-Ken

From: sumeet kumar [mailto:sumeet.kumar507 at gmail.com<mailto:sumeet.kumar507 at gmail.com>]
Sent: Monday, August 15, 2016 3:52 PM
To: Moreland, Kenneth <kmorel at sandia.gov<mailto:kmorel at sandia.gov>>
Cc: ParaView Developers <paraview-developers at paraview.org<mailto:paraview-developers at paraview.org>>
Subject: [EXTERNAL] Re: [Paraview-developers] Reading multiple files in parallel

Thanks Kenneth, for you help.
I get the vtkAlgorithm::CAN_HANDLE_PIECE_REQUEST(), set 1 part.
But, what to do next.

How can i get the piece no, so that i can update the data for it.

vtkmXMLPUnstructuredGridDataReader is not doing me a great help.

Sumeet

On Mon, Aug 15, 2016 at 12:57 PM, Moreland, Kenneth <kmorel at sandia.gov<mailto:kmorel at sandia.gov>> wrote:
I think the key you are looking for is vtkAlgorithm::CAN_HANDLE_PIECE_REQUEST(), which you want to set to 1 to signify that you can provide multiple pieces. You might want to look at vtkmXMLPUnstructuredGridDataReader for an example.

If I remember correctly, this has recently been changed in VTK. It used to be that you would give the number of pieces you supported, but that functionality was not useful in practice and consequently readers reported that they could produce any number of pieces in order to be useful. Thus, the behavior was change to allow readers simply to report whether or not they support multiple pieces. Berk can probably provide more information.

-Ken

From: Paraview-developers [mailto:paraview-developers-bounces at paraview.org<mailto:paraview-developers-bounces at paraview.org>] On Behalf Of sumeet kumar
Sent: Monday, August 15, 2016 12:58 PM
To: ParaView Developers <paraview-developers at paraview.org<mailto:paraview-developers at paraview.org>>
Subject: [EXTERNAL] [Paraview-developers] Reading multiple files in parallel

Hello all,
I have a number of files which contains information about pieces of unstructured mesh. For example :-

t_2.h5.01.feioutput
t_2.h5.03.feioutput
t_2.h5.04.feioutput
t_2.h5.05.feioutput
t_2.h5.06.feioutput
t_2.h5.07.feioutput
t_2.h5.08.feioutput
t_2.h5.09.feioutput
t_2.h5.feioutput
My reader can read each file individually but I want to read them all in parallel so that I can visualize the whole model. I looked at the http://www.paraview.org/Wiki/Writing_ParaView_Readers and followed the steps for unstructured mesh.
When I try to set the number of pieces in Request Information function as this

vtkInformation* Node_Mesh = outVec->GetInformationObject(0);
Node_Mesh->Set(vtkStreamingDemandDrivenPipeline::MAXIMUM_NUMBER_OF_PIECES(), 10);

I get the following error

error: 'MAXIMUM_NUMBER_OF_PIECES' is not a member of 'vtkStreamingDemandDrivenPipeline'
  Node_Mesh->Set(vtkStreamingDemandDrivenPipeline::MAXIMUM_NUMBER_OF_PIECES(), 10);

Also, it would be highly appreciated if anyone could show/(send me link) me to a sample example.
Regards

--
Error! Filename not specified.
Sumeet Kumar Sinha
Graduate Student
Phone: (+1) <tel:%28%2B91%29%209910516219> 5306018271<tel:5306018271>
Website : http://www.sumeetksinha.com/

--
<image001.jpg>
Sumeet Kumar Sinha
Graduate Student
Phone: (+1) <tel:%28%2B91%29%209910516219> 5306018271<tel:5306018271>
Website : http://www.sumeetksinha.com/

--
[http://faculty.engineering.ucdavis.edu/ceetemplate/wp-content/uploads/2013/06/CEE75percent500w.jpg]
Sumeet Kumar Sinha
Graduate Student
Phone: (+1) <tel:%28%2B91%29%209910516219> 5306018271
Website : http://www.sumeetksinha.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview-developers/attachments/20160816/cdf36371/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 407 bytes
Desc: image001.jpg
URL: <http://public.kitware.com/pipermail/paraview-developers/attachments/20160816/cdf36371/attachment-0001.jpg>