[Paraview] Problem with custom time-aware reader

[Karl König]

Hi Ken and Stéphane,

I see your point about ignoring duplicate times in a file series in case
of overlapping restarted simulations. I actually know that condition
very well and usually resort to manually deleting the duplicate output
files. Admittedly, that is possible for a file format with one time step
per file (and very easy to realise in case it is (compressed) ASCII
format), but ParaView supports other cases as well (e.g. ExodusII file
format). I didn't have that use case in mind and you're right, my
suggested patch would interfere with it.

I had thought about the hack-ish solution you suggested earlier, but to
be on the safe side I'd have to inspect to complete file series first,
before being able to choose a safe delta depending on the size of time
steps found. Not willing to duplicate the class FileSeriesReader any
more (or misusing it like I did in my initial posting and using my own
file name book keeping to choose the correct file name in RequestData
depending on the time value given), but relying on it to call my
reader's RequestInformation for all files by now, I'd have to tell
FileSeriesReader after having read all files to re-adjust time steps.
But there are two problem with that approach: a) How to detect the last
file of a series from within my reader? I'd have to patch
FileSeriesReader for that, because such information is not passed up to
now as far as I can tell. b) How to manipulate the variable Internal (of
structure vtkFileSeriesReaderInternals) being private in
FileSeriesReader from within my reader without the need to reread the
file series?

The second suggestion is not really an option as I'd like to support
both cases: proper disjoint time steps and the case where all time step
values are identical. The patch I suggested actually does more than I
need. One could limit it to the case that all files contain the same
time step instead of checking more generally whether there are more
files than time steps.

The third suggestion, a flag in your reader to ignore time, is
interesting. With a bit of extra XML like
     <IntVectorProperty name="Ignore time values"
            command="SetFoo"
            number_of_elements="1"
            animateable="0"
            default_values="0">
       <BooleanDomain name="bool"/> // GUI object type: Check box
       <Documentation>
         Toggle whether to respect the time step values found in files.
         If checked, make sure that time step values are disjoint.
         Otherwise it will not be possible to step through all files,
         but be limited to last files of every series subset with
         identical time step values.
       </Documentation>
     </IntVectorProperty>
I could add a check box to the GUI, but by the time the check box is
displayed, RequestInformation has been called already and enabling the
check box such that stored time steps are ignored would mean facing the
problem b) again: Either trigger a costly reread of the complete file
series and instruct my reader to not report time step values this time
or patch FileSeriesReader to somehow be able to manipulate the variable
Internal from within my reader after having read all files.
It would be desirable to be able to make the choice before
RequestInformation is called. But from a recent discussion
(http://www.paraview.org/pipermail/paraview/2009-September/013524.html)
I learned that making choices beforehand is not possible.

I find Stephane's email very interesting: how did you realise your
TimeMode={PhysicalTime,Iteration} setting? You do not rely on
FileSeriesReader, do you? Did you clone it and integrate it into your
reader? Such that you have full control over the mapping file name <=>
time steps from within your reader?

Karl


On 9/11/09 00:44 AM, Kenneth Moreland wrote:

> Actually, we designed the file series reader with the understanding that
> time ranges might overlap, but with a different use case in mind.  Our
> users often have to run simulations far past the mean time to failure of
> the platforms they are running on.  To handle the problem of the
> inevitable crash of the system, the simulation periodically writes out
> checkpoint files.  When the system fails, the simulation is “backed
> back” to the last checkpoint and restarted.  As the simulation reruns
> from this point, it will recompute some time steps and rewrite out the
> data for those times.  The file series reader understands that these
> time steps are duplicated and picks the one most likely to be correct.
> 
> So the problem is that your case excludes this case.  If the file series
> reader declared an error whenever it encountered multiple versions of
> the same time, it would not be reading in the restarted simulation data
> correctly.  The problem is, your case does not really represent an
> actual time series in the physical sense.  I can see the utility of
> looking at the intermediate solutions, but these intermediate solutions
> are not a real progression of events in a physical sense.  It is by
> removing the intermediate results that you get a progression of actual
> physical events.  In fact, I can see utility in viewing the data in
> either way.
> 
> So instead, I suggest some workarounds.  First, your reader might be
> able to sense which iterative intermediate result it is reading and then
> add some delta to the time to make the file series reader recognize them
> as different times.  This is probably the most hack-ish, as, again,
> these intermediate results do not really represent a time series (even
> if you want them animated as such).  Second, you could have a flag in
> your reader to ignore time.  If your reader does not set the time keys,
> the file series reader will treat the files as an ordinary sequence.
>  Third, you (or I) could add a flag to the file series reader that would
> force it to ignore the time in the reader.  This flag would be hidden
> for most readers, but could be exposed for your reader.
> 
> -Ken