[Paraview] Problem with custom time-aware reader

Sun Sep 6 17:24:05 EDT 2009

Hi Ken,

Thank you very much for taking the time to look into my problem again.
As a matter of fact I had worked out a solution using its own
bookkeeping of files by means of a "vtkStringArray *FileNames".
But you are right, your solution is minimal and avoids re-implementing
features already provided by
ParaView3/Servers/Filters/vtkFileSeriesReader.cxx, features I haven't
been aware of because I simply followed the instructions to use the
class without actually studying it.

I hope you don't mind that I still have a few questions about the
prototypical time-aware reader FooReader for files series with one time
step value per file and no meta file.

1) I opened the file series file*.foo from the tarball attached to my
initial posting and opened the Python shell from within ParaView 3.6.1
(having successfully loaded the plugin by setting PV_PLUGIN_PATH). I get
a warning I cannot make any sense of:

Python 2.5.2 (r252:60911, Jan  4 2009, 18:00:02)
[GCC 4.3.2] on linux2
>>> from paraview.simple import *
Warning: FooReader is being overwritten. This may point to an issue in
the ParaView configuration files
>>> 

Deleting ~/.config/ParaView/ParaView3.6.ini does not remove the warning.
 I wonder what's wrong with this reader plugin as for the other reader
plugin that comes with ParaView 3.6.1, the VisIt Reader, there is no
such warning.
I get the same message for a cvs build (paraview version 3.7.0, Date:
2009-09-05).

2) Despite the warning, using the reader works from Python
>>> view = GetActiveView(); view.ViewTime; 
0.0
>>> reader = GetActiveSource(); reader.TimestepValues
[0.0, 0.10000000000000001, 0.20000000000000001, 0.29999999999999999,
0.40000000000000002, 0.5, 0.59999999999999998, 0.69999999999999996,
0.80000000000000004, 0.90000000000000002]
>>> view.ViewTime = -1.2; Render()
=> on standard output I can see that RequestData is invoked and the
first file, file0.foo, is loaded, i.e. clamping provided by
FileSeriesReader works for FooReader. Same for snapping to files for
non-existent time step values works:
>>> view.ViewTime = 0.19; Render()
=> second file, file1.foo, is loaded.

But I noticed from the standard output that only RequestData is called.
When using the VCR controls from the GUI to step to the next file,
ParaView behaves differently: first RequestInformation is called, then
RequestData, then again RequestInformation. If both RequestInformation
and RequestData contained a call to MyReadFile, that would be three
times parsing the same file.
a) Could someone please elaborate on the need to calling
RequestInformation twice?
b) Would I run into trouble (e.g. when using some kind of filter) if I
kept book of the file read last and skipping RequestInformation if
this->FileName hasn't changed since called last time? That would
eliminate the overhead of the trailing RequestInformation call.
c) Would I run into trouble if I kept book of all files read initially
(i.e. after pressing ok in the open file dialog) and completely skipped
later calls of RequestInformation, e.g. via

int vtkFooReader::RequestInformation(vtkInformation *request,
vtkInformationVector **inputVector, vtkInformationVector *outputVector)
{
  if (this->FileNames->LookupValue(this->FileName) < 0)
    {
    ...
    this->FileNames->InsertNextValue(this->FileName);
    ...
    }
  return 1;
}

I tried b) and c) and applying a couple of filters I did not notice any
unwanted side effects. I couldn't test all of them, so did I miss a
combination where skipping RequestInformation before and after calling
RequestData - but still having run RequestInformation once for all files
initially - does make a difference?

Thank you very much in advance.

Karl

----- Original Message -----
From: "Moreland, Kenneth" <kmorel at sandia.gov>
To: Karl König <kkoenig11 at web.de>
CC: "paraview at paraview.org" <paraview at paraview.org>
Sent: Sonntag, 6. September 2009 18:20:26
Subject: [Paraview] Problem with custom time-aware reader
> OOPS!  STOP!  BACK UP!  I looked at your code too quickly, misunderstood
> what you were doing, and gave totally the wrong advise.  Please ignore
> everything I said before.
> 
> If you are using the file series reader, then your reader should be
> completely ignorant of any file series.  It should read in only the file
> it is given, and if that file changes then it should ignore whatever
> file it was previously given.  Therefore, the problem with your reader
> is that it is trying to collect time information over all the files.
>  That is the job of the file series reader and as a result it is fouling
> up the operation of the file series reader.
> 
> So, what your reader should do is read in the time value in the file it
> is given, set the TIME_STEPS key to ONLY that time value and set the
> TIME_RANGE to be only that range.  Attached is a modified version of
> your reader example that has all that time series management stripped
> out.  The resulting code is much smaller and actually works.
> 
> Specifically what was happening was that by the time RequestInformation
> was called on the last time step, your reader had collected information
> about all the time steps and returned all the time steps in all the
> files.  The file series reader thought you meant that the last file
> contained all those time steps (some file formats do contain multiple
> time steps in a single file).  Because your reader said that the last
> file contained all the time steps, the file series reader was using that
> last file for all the time steps.
> 
> -Ken
> 
> 
> On 9/3/09 11:24 PM, "Karl König" <kkoenig11 at web.de> wrote:
> 
>     Hi Ken,
> 
>     Thanks again for your input.
> 
>     > You have the basic idea.  The seg fault is probably happening because
>     > the destructor of you class is trying to free the pointer you set
>     to it,
>     > which is probably actually pointing to some spot on the stack.
>     >
>     > You are probably not seeing this mapping/lookup in the VTK IO classes
>     > because you are looking at classes that do not directly support time
>     > (such as the legacy readers and XML readers).  Those readers read
>     > exactly one file with one time step in it.  ParaView has a magic meta
>     > reader called a FileSeriesReader that takes a real reader and a
>     > collection of files and multiplexes the files to the reader based
>     on the
>     > time.
>     >
>     > In retrospect, this is probably an easier way to go (assuming your
>     final
>     > reader is a file series like this).  Documentation on using the
>     > FileSeriesReader is on the Wiki at
>     >
>     >     http://www.paraview.org/Wiki/Restarted_Simulation_Readers#Customized_Restart_Reader
> 
>     I've been aware of that page. In fact, FooReader.xml and
>     FooReaderGUI.xml of the tarball I posted already did make use of the
>     FileSeriesReader. Together with calling
>       outInfo->Set(vtkStreamingDemandDrivenPipeline::TIME_STEPS(), ...)
>       outInfo->Set(vtkStreamingDemandDrivenPipeline::TIME_RANGE(), ...)
>     in RequestInformation it is responsible for inspecting all files of the
>     series between selecting them in the file open dialog and appearing of
>     the Apply button.
> 
>     Anyway, I'll try to use again the mapping time step value -> file name
>     and iron out the segfault on deletion of the reader object.
> 
>     I'll post the solution in case I can work it out and someone is
>     interested.
> 
>     Karl
> 
> 
> 
> 
> 
>    ****      Kenneth Moreland
>     ***      Sandia National Laboratories
> ***********  
> *** *** ***  email: kmorel at sandia.gov
> **  ***  **  phone: (505) 844-8919
>     ***      web:   http://www.cs.unm.edu/~kmorel
> 