VTK/Composite data changes

Primarily motivated by better support of AMR data structures, various APIs related to vtkCompositeDataSet have been changed.

The New vtkCompositeDataSet

In previous versions, vtkCompositeDataSet represents a tree structure that stores the actual data sets at its leaves. Most usage of it, on the other hand, treats it simply as a data set container that can be traversed. So we extracted the set traversal APIs to form a fully abstract class, again named vtkComposisteDataSet. The original vtkCompositeDataSet is renamed vtkDataObjectTree. The refactored hierarchy is shown below. Note that in this hierarchy, vtkUniformGridAMR switches to a new implementatation (See the third section for details).

As a consequence of the change, the class vtkCompositeDataIterator is refactored in a similary fashion: Those APIs that support simple traverals form the new vtkCompositeDataIterator; the original class is is renamed vtkDataObjectTreeIterator.

Demand Driven Composite Data Loading

One strategy to visualize large data sets is to make the data loading demand driven: A data source only produces the data blocks that downstream filters ask for. We now provide the support for composite data through two pipeline keys:

• vtkCompositeDataPipeline::COMPOSITE_DATA_META_DATA(), created by a data

source to provide meta data for downstream filters.

• vtkCompositeDataPipeline::UPDATE_COMPOSITE_INDICES(), created by

downstream filters to request blocks from upstream data sources.

Example

We can look at how a vtkAMRBaseReader source works with a vtkCompositeCutter filter in a demand driven pipeline. At the request information pass, a vtkAMRBaseReader creates a meta data object that has the same type as the output but no actual data blocks:

<source lang="cpp"> int vtkAMRBaseReader::RequestInformation(... ...)

 ... ...
 this->Metadata = vtkOverlappingAMR::New();
 this->FillMetaData( ); //where file format specific subclasses would fill the meta data
 vtkInformation* info = outputVector->GetInformationObject(0);
 info->Set( vtkCompositeDataPipeline::COMPOSITE_DATA_META_DATA(), this->Metadata );
 ... ...

</source>

At the RequestUpdateExtent pass, vtkCompositeCutter iterates over the bounding boxes of all data blocks, determines which ones are needed and sends their indices upstream.

<source lang="cpp"> int vtkCompositeCutter::RequestUpdateExtent(... ...)

 ... ...
 std::vector<int> intersected;
 vtkCompositeDataSet * meta= vtkCompositeDataSet::SafeDownCast(inInfo->Get(vtkCompositeDataPipeline::COMPOSITE_DATA_META_DATA()));
 vtkSmartPointer<vtkCompositeDataIterator> iter;
 iter.TakeReference(meta->NewIterator());
 iter->SetSkipEmptyNodes(false);
 for(iter->InitTraversal(); !iter->IsDoneWithTraversal(); iter->GoToNextItem())
   {
   double* bb = iter->GetCurrentMetaData()->Get(vtkDataObject::BOUNDING_BOX());
   for (int c=0; c < this->ContourValues->GetNumberOfContours(); c++)
     {
     if(IntersectBox(this->GetCutFunction(),bb,this->ContourValues->GetValue(c)))
       {
       intersected.push_back(iter->GetCurrentFlatIndex());
       break;
       }
      }
   }
 inInfo->Set(vtkCompositeDataPipeline::UPDATE_COMPOSITE_INDICES(), &intersected[0], static_cast<int>(intersected.size()));

... ... </source>

Note that vtkCompositeCutter does not down cast the meta data "all the way" to vtkOverlappingAMR but only to vtkCompositeDataSet. This is designed so that this filter would work for any composite data set that provides bounding box information through its iterator.

Finally, in the RequestData pass, the amr source would load the exactly the requested blocks using the key vtkCompositeDataPipeline::UPDATE_COMPOSITE_INDICES().

New AMR Data Structures

The previous implementation of vtkUniformGridAMR uses data structures provided by the parent class (the old vtkCompositeDataSet). The implementation is inefficient when there are hundreds of thousands of data blocks, which is unfortunately common for AMR. In the new implementation, we use more compact data structures: The following two members of vtkUniformGridAMR respectively stores the meta data and the actual data: <source lang="cpp"> vtkAMRInformation* AMRInfo; vtkAMRDataInternals* AMRData; </source>

There are two children classes of vtkUniformGridAMR: vtkOverlappingAMR and vtkNonOverlappingAMR. vtkOverlappingAMR is designed for AMR file types that provide meta data, such as Enzo or Flash, so it provides access to both AMRInfo and AMRdata; vtkNonOverlappingAMR is designed for AMR file types that do not contain meta data, such as Chmobo, so it provides access to only AMRData.