[Paraview] Assistance/Suggestion With Animating Relatively Small Datasets Over Many Timesteps

Thu Mar 22 17:42:08 EDT 2012

Ah yes,
    This is the sort of approach I was initially hoping to get 
implemented.  However I am still really stuck on how to implement some of 
those details.

A couple questions I am still trying to hammer out specifically:
  - You say "create a RequestData function"; by this do you mean the 
'RequestInformation Script' field inside of a ParaView ProgrammableFilter?
  - I have searched for longer than probably is justified for this 
already....where does Python have access to an UPDATE_TIME field?
  - How do I place the Python script in a way that it is evaluated every 
single time the ParaView UI updates it's own timestep - otherwise the 
filter won't update during an animation tick

I appreciate the response - if you have any additional thoughts on this 
I'd love to hear about them in further detail.

Best wishes!
     -Another John

From:
"Biddiscombe, John A." <biddisco at cscs.ch>

To:
John CTR Mick/ACT/CNTR/FAA at FAA, "paraview at paraview.org" 
<paraview at paraview.org>
Date:
03/22/2012 04:57 PM
Subject:
RE: [Paraview] Assistance/Suggestion With Animating Relatively Small 
Datasets Over Many Timesteps

I would stick to the CSV file if you?ve got all the information you need 
inside it

? but instead of using a table to points filter, create your own custom 
filter which takes the CSV table as input, and outputs just a small 
polydata object which has the current positions (+scalars) encoded for the 
given time.

All you need to do is create a RequestData function which looks at 
UPDATE_TIME, and fetches the corresponding rows/columns for each 
planet/point for that time and adds them to the output.

For added fun, you can export a multi-block dataset, and/or have a 
treeview which allows you to enable/disable certain planets/points. This 
way you can

Reader -> ExtractPlanetFilter1 ->Render (use textureA)
                    ExtractPlanetFilter2->Render (use tectureB)

[Or other combinations with ExtractBlockFilter if you use multiblock ].

The Reader would simply subclass polydataalgorithm, but take a vtkTable as 
input. Couple of hours work is all you?d need and you can leave the data 
in its current form.

$0.02

JB

From: paraview-bounces at paraview.org [mailto:paraview-bounces at paraview.org] 
On Behalf Of John.CTR.Mick at faa.gov
Sent: 22 March 2012 17:13
To: paraview at paraview.org
Subject: [Paraview] Assistance/Suggestion With Animating Relatively Small 
Datasets Over Many Timesteps

     This is my first experience posting to the ParaView Mailing List; as 
well as my first project I am exploring with ParaView.  I hope I am 
posting this question to the correct audience. 

     I am attempting to create an animation of our Solar System - at an 
extremely high update interval - perhaps plotting the position and 
velocity of significant solar system bodies over 165 Earth years at 
a one day interval.  Initially I thought I would be able to work with CSV 
files to define a dataset which includes time, positional coordinates, and 
velocity vectors.  While testing this approach I managed to use the 
TableToPoints filter to create a visualization of a sample dataset; but I 
was not able to sequence over the individual timesteps.  So in effect you 
see all the positions a particular planet was at for the entire duration 
of the dataset. 

I created a video of this test and made it available online: 
     http://www.youtube.com/watch?v=XLPxx-2Zc9Q 

     Next I thought perhaps the CSV format was a little too general for 
what I needed, so I read the VTK file format document and migrated over to 
the XML VTK format.  It seems like there is no way to define time 
sequenced data natively within the file format itself?  ParaView expects 
the user to break the timesteps into individual files - with a numeric 
indicator in the filename to define the sequence.  If I were to take this 
approach, I could be looking at creating nearly 500,000 individual files 
to represent the entire duration of time I'm interested in plotting.  I 
feel like there must be a better way to do this. 

     As a small improvement I thought I could at least combine all of the 
unique solar system bodies (Earth, Sun, etc.), into a single timestep file 
such as this: 
          (filename: timeseries_2455562.vtu) 

<?xml version="1.0"?> 
<VTKFile type="UnstructuredGrid" version="0.1" byte_order="LittleEndian" 
compressor="vtkZLibDataCompressor"> 
  <UnstructuredGrid> 
    <Piece NumberOfPoints="2" NumberOfCells="2"> 
      <PointData Vectors="Velocity" Scalars="Radius"> 
        <DataArray Name="Time" type="Float32" format="ascii">2455562 
2455562</DataArray> 
        <DataArray Name="Radius" type="Float32" format="ascii">6371 
696000</DataArray> 
        <DataArray Name="Velocity" type="Float32" NumberOfComponents="3" 
format="ascii"> 
          -2.982651603443954E+01 -5.304542013817379E+00 
-7.226846229160870E-04 
          0 0 0 
        </DataArray> 
      </PointData> 
      <Points> 
        <DataArray type="Float32" NumberOfComponents="3" format="ascii"> 
          -2.567030301930377E+07 1.448508985971261E+08 
-3.607762965630511E+03 
          0 0 0 
        </DataArray> 
      </Points> 
      <Cells> 
        <DataArray Name="connectivity" type="Int32" format="ascii">0 
1</DataArray> 
        <DataArray Name="offsets" type="Int32" format="ascii">1 
2</DataArray> 
        <DataArray Name="types" type="UInt8" format="ascii">1 
1</DataArray> 
      </Cells> 
    </Piece> 
  </UnstructuredGrid> 
</VTKFile> 

The idea of this approach was to list all of the scalars and vectors for 
each corresponding body in a single file.  In this example the first 
listed scalar, vector, or coordinate point, is the Earth while the second 
set of values indicates the Sun's data.  By combining the bodies into a 
single file I could reduce the 500,000 files required down to some 60,000 
files....still undesirable but better. 

However I would also like to segment the Points once inside the ParaView 
environment; so I can create a distinct pipeline for each body in the 
dataset.  One application for this would be to allow different Glyph 
rendering rules for the Sun and the Earth so I can make the Sun...have a 
Sun texture - and the Earth an Earth texture. 

After experimenting with this for awhile I found myself trying to 
implement a Programmable Filter.  The filter would take the Unstructured 
Grid for the given timestep, and given an index, extract the Point, 
Scalar, and Vector data from its input - then set only those values to the 
Output Unstructured Grid.  I currently have a script that looks something 
like this: 

# CELLID 0 = EARTH 
cellId = 0 

# Obtain References To Input/Output Data 
gridI = self.GetUnstructuredGridInput() 
gridO = self.GetUnstructuredGridOutput() 
dataI = self.GetInput() 
dataO = self.GetOutput() 
scalarI = dataI.GetPointData().GetScalars() 
vectorI = dataI.GetPointData().GetVectors() 

# Initialize Local Storage Arrays 
newVector = vtk.vtkFloatArray() 
newVector.SetName(vectorI.GetName()) 
newVector.SetNumberOfComponents(3) 
newScalar = vtk.vtkFloatArray() 
newScalar.SetName(scalarI.GetName()) 
newPoint = vtk.vtkPoints() 
newPoint.SetNumberOfPoints(1) 
newCell = vtk.vtkCellArray() 

# Lookup CellId References 
idCell = gridI.GetCell(cellId) 
idCellType = gridI.GetCellType(cellId) 
idPoint = gridI.GetPoints().GetPoint(cellId) 
idScalar = scalarI.GetTuple(cellId) 
idVector = vectorI.GetTuple(cellId) 

# Populate New Local Cell Array with CellId Cell 
newCell.InsertNextCell(idCell) 

# Populate New Local Point Array With CellId Point 
newPoint.SetPoint(0, idPoint) 

# Populate New Local Scalars with CellId Scalar Tuples 
newScalar.InsertNextTuple(idScalar) 

# Populate New Local Vectors with CellID Vector Tuples 
newVector.InsertNextTuple(idVector) 

# Copy Local Cells, Scalars, Vectors, and Points Into Output Pipeline Data 

gridO.SetPoints(newPoint) 
gridO.SetCells(idCellType, newCell) 
dataO.GetPointData().SetScalars(newScalar) 
dataO.GetPointData().SetVectors(newVector) 

The filter does achieve the desired effect for the cellID 0 - but when I 
switch the value of cellID to 1 (which should refer to the Sun's index) 
ParaView crashes hard - where I assume I am missing something, possibly 
requesting something out of bounds in memory. 

I've been struggling with this problem for a little while now and thought 
to share my progress here; hopefully there is somebody out there with a 
little more insight that could help guide me towards either a conclusion 
with this implementation or a suggestion for a better approach. 

Essentially - How can one give ParaView Point and Velocity vectors over a 
vast sequence of time without creating potentially hundreds of thousands 
of files on disc?  Does this Programmable Filter seem like a reasonable 
approach?  Is there a native feature I am missing that does this already? 
Is there some other file format other than VTK I could use to have 
ParaView understand this format of data? 

Any thoughts, comments, or questions will be greatly appreciated.  :) 

Thanks, 
     John 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20120322/cc564944/attachment-0001.htm>