[Paraview] Some D3 and MPI questions

[John Biddiscombe]

Ken
> Another solution is to use the ability of vtkKdTree to partition several
> data sets at the same time.  
I believe the correct expression is "you da man!" - this is perfect. It 
works just how I wanted it to. Thanks for the help. The entire dataset 
is partitioned using the initial cuts and I get the output I expected. 
Blocks are not subdivided individually, but rather the entire domain is 
partitioned.

Having got it working, I can see that I may need to modify d3 to allow a 
AddDataSet(...) in a similar manner to the KdTree to make it cleaner, 
and from there, it's only a small step to produce a 
vtkMultiGroupDistributeData filter which would do the distribution on a 
multiblock data using the global partition, instead of splitting each 
block individually between processors.

For reference, here's an animation which shows the new processor 
boundary - it is no longer along the edge of the rotating doman, but 
further along to the right - the dataset is very long and thin and this 
only whows the left end. Having the new domain include the rotating 
edges all inside one processor is going to help debug significantly.
NB. The particles were supposed to change colour as the cross the 
border, but I messed up, so you'll have to just imagine it :) 
ftp://ftp.cscs.ch/out/biddisco/flowviz/D3_Particles_3.avi
(NB2 some particles disappear as they cross the rotating edge - this is 
caused by tiny cracks between the two meshes where they don't match up 
properly. I'm cogitating what to do about it)

Thanks again. (All I need now is to get offscreen rendering working and 
I'll be ready for anything).

JB




> First, create your own vtkPKdTree object
> (separate from that in D3).  Second, read in all of the data for each
> time step and add the vtkDataSet from each block to your vtkPKdTree
> (that should be easy because you are in the reader, right?).  Third,
> copy the structure of your vtkPKdTree to D3's k-d tree
> (D3->GetKdTree()->SetCuts(mytree->GetCuts()).  This should happen before
> D3's first execution.  Now you should be ready to process any timesteps.
>
> With this mode you will have one global partitioning that is applied to
> all of the blocks.  Thus, each block taken independently will have an
> uneven partition, but all the blocks together will have an even
> partition.
>
> -Ken
>
>   
>> -----Original Message-----
>> From: John Biddiscombe [mailto:biddisco at cscs.ch]
>> Sent: Monday, November 20, 2006 11:54 AM
>> To: Moreland, Kenneth
>> Cc: Paraview Mailing List
>> Subject: Re: [Paraview] Some D3 and MPI questions
>>
>> That's it - because it's multiblock - I'm only getting the Cuts from
>>     
> the
>   
>> last pass of the CompositeDataPipeline loop - so the planar slice is
>> correct, but only for the final part.
>>
>> Damn. How can I get the BSP cuts for all parts of the multiblock data
>>
>> JB
>>
>>     
>>> Ken,
>>>
>>>       
>>>> I'm a little confused.  Once you call SetCuts on a k-d tree, the
>>>> distribution should be set and will never be recalculated.  Are you
>>>> calling D3->GetKdTree()->SetCuts()?
>>>>
>>>>         
>>> On the animation I posted, I was not re-using the cuts, I was
>>>       
> allowing
>   
>>> the tree to be recomputed on each timestep.
>>>
>>> When I tried to re-use the cuts, as below - the first time the
>>>       
> reader
>   
>>> executes, we have this->FirstTime=true and the Cuts are saved, then
>>> reused on subsequent passes - but the first frame looks ok and the
>>>       
> ones
>   
>>> after look very dodgy.
>>>
>>> The data is Multiblock and the D3 filter accepts only Unstructured,
>>>       
> so I
>   
>>> pass the data through a multigroupscalar filter and force the pieces
>>> manually to trigger the copmposite data pipeline into looping for me
>>>       
> and
>   
>>> generating multiblock D3 output (it works very nicely).
>>> The first execution gives the "right" looking data, but after that
>>>       
> it
>   
>>> changes and looks less well balanced. I concluded that it was
>>>       
> storing
>   
>>> not just the cuts, but also some Cell IDs and since these were
>>>       
> invalid
>   
>>> onsubsequent time steps, the thing failed to generate the proper
>>>       
> output.
>   
>>> It does seem to work consistently after the second execution, ie it
>>>       
> does
>   
>>> not change as you correctly state, but it doesn't look so good
>>>       
> either -
>   
>>> and why does it change the second time. I'm checking to see if I've
>>>       
> made
>   
>>> a boo-boo somewhere, but I welcome help.
>>>
>>> cheers
>>>
>>> JB
>>>
>>>
>>> #ifdef USE_D3
>>>    if (this->updateNumPieces>1) {
>>>      //
>>>      // Before doing any D3 filter operation, set a barrier point to
>>> synchronize things
>>>      //
>>>      this->Controller->Barrier();
>>>
>>>      vtkCompositeDataPipeline *cp = vtkCompositeDataPipeline::New();
>>>      this->D3->SetDefaultExecutivePrototype(cp);
>>>      this->scalars->SetDefaultExecutivePrototype(cp);
>>>      cp->Delete();
>>>
>>>      vtkStreamingDemandDrivenPipeline* exec =
>>> vtkStreamingDemandDrivenPipeline::SafeDownCast(this->D3-
>>> GetExecutive());
>>>      exec->SetUpdateNumberOfPieces(exec->GetOutputInformation(0),
>>> this->updateNumPieces);
>>>      exec->SetUpdatePiece(exec->GetOutputInformation(0),
>>> this->updatePiece);
>>>
>>>      if (!this->FirstStep && this->BSPcuts) {
>>>        this->D3->GetKdtree()->SetCuts(this->BSPcuts);
>>>        CSCSOutputMacro(<<"Using BSPCuts from previously ");
>>>      }
>>>      this->scalars->SetInput(mb);
>>>      this->D3->SetInputConnection(this->scalars->GetOutputPort());
>>>      this->D3->SetController(this->Controller);
>>>      this->D3->SetBoundaryModeToAssignToAllIntersectingRegions();
>>>      this->D3->UseMinimalMemoryOff();
>>>      this->D3->SetRetainKdtree(1);
>>>      exec->Update();
>>>      if (this->FirstStep) {
>>>        if (this->BSPcuts) this->BSPcuts->Delete();
>>>        this->BSPcuts = this->D3->GetKdtree()->GetCuts();
>>>        this->BSPcuts->Register(this);
>>>        this->BSPcuts->PrintSelf(std::cout, 0 );
>>>        this->FirstStep = 0;
>>>      }
>>>      //
>>>      //
>>>      //
>>>      dob = exec->GetOutputData(0);
>>>    }
>>> #endif
>>>
>>>
>>> _______________________________________________
>>> ParaView mailing list
>>> ParaView at paraview.org
>>> http://www.paraview.org/mailman/listinfo/paraview
>>>       
>> --
>> John Biddiscombe,                            email:biddisco @ cscs.ch
>> http://www.cscs.ch/about/BJohn.php
>> CSCS, Swiss National Supercomputing Centre  | Tel:  +41 (91) 610.82.07
>> Via Cantonale, 6928 Manno, Switzerland      | Fax:  +41 (91) 610.82.82
>>
>>     
>
>
>   


-- 
John Biddiscombe,                            email:biddisco @ cscs.ch
http://www.cscs.ch/about/BJohn.php
CSCS, Swiss National Supercomputing Centre  | Tel:  +41 (91) 610.82.07
Via Cantonale, 6928 Manno, Switzerland      | Fax:  +41 (91) 610.82.82