[Paraview] Problem with custom time-aware reader
Moreland, Kenneth
kmorel at sandia.gov
Thu Sep 10 18:44:20 EDT 2009
Actually, we designed the file series reader with the understanding that time ranges might overlap, but with a different use case in mind. Our users often have to run simulations far past the mean time to failure of the platforms they are running on. To handle the problem of the inevitable crash of the system, the simulation periodically writes out checkpoint files. When the system fails, the simulation is "backed back" to the last checkpoint and restarted. As the simulation reruns from this point, it will recompute some time steps and rewrite out the data for those times. The file series reader understands that these time steps are duplicated and picks the one most likely to be correct.
So the problem is that your case excludes this case. If the file series reader declared an error whenever it encountered multiple versions of the same time, it would not be reading in the restarted simulation data correctly. The problem is, your case does not really represent an actual time series in the physical sense. I can see the utility of looking at the intermediate solutions, but these intermediate solutions are not a real progression of events in a physical sense. It is by removing the intermediate results that you get a progression of actual physical events. In fact, I can see utility in viewing the data in either way.
So instead, I suggest some workarounds. First, your reader might be able to sense which iterative intermediate result it is reading and then add some delta to the time to make the file series reader recognize them as different times. This is probably the most hack-ish, as, again, these intermediate results do not really represent a time series (even if you want them animated as such). Second, you could have a flag in your reader to ignore time. If your reader does not set the time keys, the file series reader will treat the files as an ordinary sequence. Third, you (or I) could add a flag to the file series reader that would force it to ignore the time in the reader. This flag would be hidden for most readers, but could be exposed for your reader.
-Ken
On 9/9/09 8:46 AM, "Karl König" <kkoenig11 at web.de> wrote:
Hi Ken,
Sorry to persist, but I found a case that is not properly dealt with by
the class FileSeriesReader in cooperation with the reader you kindly
modified helping me out:
There are cases, though rare, where multiple files can contain identical
time step values. Think about a transient nonlinear solution process
where the linear solver exports its intermediate solutions to disk - all
files within one nonlinear step will contain the same time step value.
There are probably other cases as well.
By fixing bug 8892, you added (among others) these instructions to
Servers/Filters/vtkFileSeriesReader.cxx:
> int vtkFileSeriesReaderTimeRanges::GetAggregateTimeInfo([...])
> {
> [...]
> double timeRange[2];
> timeRange[0] = this->RangeMap.begin()->second
> ->Get(vtkStreamingDemandDrivenPipeline::TIME_RANGE())[0];
> timeRange[1] = (--this->RangeMap.end())->second
> ->Get(vtkStreamingDemandDrivenPipeline::TIME_RANGE())[1];
>
> // Special case: if the time range is a single value, supress it. This is
> // most likely from a data set that is a single file with no time anyway.
> // Even if it is not, how much value added is there for a single time value?
> if (timeRange[0] >= timeRange[1])
> {
> outInfo->Remove(vtkStreamingDemandDrivenPipeline::TIME_RANGE());
> outInfo->Remove(vtkStreamingDemandDrivenPipeline::TIME_STEPS());
> return 1;
> }
which lead to the situation that only the last file of a file series is
loaded by readers depending on class FileSeriesReader if the files
contain non-unique time step values. You can test it with the attached
reader plugin and the file series sampleinputfiles/sametimes*.foo.
You'll notice on STDOUT that only sametimes9.foo, the last file, is loaded.
I'd like to propose a patch that warns the user for such a file series
and falls back to fake time step values. I'm not sure about implications
for animation functions, though, I'm afraid. But having applied the
patch, one can step through sametimes*.foo as well.
(Actually, willing to support the same-time-step-values-in-all-files
case is the reason why I went through the trouble to have my own book
keeping in my reader.)
Karl
----- Original Message -----
From: "Moreland, Kenneth" <kmorel at sandia.gov>
To: Karl König <kkoenig11 at web.de>
CC: "paraview at paraview.org" <paraview at paraview.org>
Sent: Sonntag, 6. September 2009 18:20:26
Subject: [Paraview] Problem with custom time-aware reader
> OOPS! STOP! BACK UP! I looked at your code too quickly, misunderstood
> what you were doing, and gave totally the wrong advise. Please ignore
> everything I said before.
>
> If you are using the file series reader, then your reader should be
> completely ignorant of any file series. It should read in only the file
> it is given, and if that file changes then it should ignore whatever
> file it was previously given. Therefore, the problem with your reader
> is that it is trying to collect time information over all the files.
> That is the job of the file series reader and as a result it is fouling
> up the operation of the file series reader.
>
> So, what your reader should do is read in the time value in the file it
> is given, set the TIME_STEPS key to ONLY that time value and set the
> TIME_RANGE to be only that range. Attached is a modified version of
> your reader example that has all that time series management stripped
> out. The resulting code is much smaller and actually works.
>
> Specifically what was happening was that by the time RequestInformation
> was called on the last time step, your reader had collected information
> about all the time steps and returned all the time steps in all the
> files. The file series reader thought you meant that the last file
> contained all those time steps (some file formats do contain multiple
> time steps in a single file). Because your reader said that the last
> file contained all the time steps, the file series reader was using that
> last file for all the time steps.
>
> -Ken
>
>
> On 9/3/09 11:24 PM, "Karl König" <kkoenig11 at web.de> wrote:
>
> Hi Ken,
>
> Thanks again for your input.
>
> > You have the basic idea. The seg fault is probably happening because
> > the destructor of you class is trying to free the pointer you set
> to it,
> > which is probably actually pointing to some spot on the stack.
> >
> > You are probably not seeing this mapping/lookup in the VTK IO classes
> > because you are looking at classes that do not directly support time
> > (such as the legacy readers and XML readers). Those readers read
> > exactly one file with one time step in it. ParaView has a magic meta
> > reader called a FileSeriesReader that takes a real reader and a
> > collection of files and multiplexes the files to the reader based
> on the
> > time.
> >
> > In retrospect, this is probably an easier way to go (assuming your
> final
> > reader is a file series like this). Documentation on using the
> > FileSeriesReader is on the Wiki at
> >
> > http://www.paraview.org/Wiki/Restarted_Simulation_Readers#Customized_Restart_Reader
>
> I've been aware of that page. In fact, FooReader.xml and
> FooReaderGUI.xml of the tarball I posted already did make use of the
> FileSeriesReader. Together with calling
> outInfo->Set(vtkStreamingDemandDrivenPipeline::TIME_STEPS(), ...)
> outInfo->Set(vtkStreamingDemandDrivenPipeline::TIME_RANGE(), ...)
> in RequestInformation it is responsible for inspecting all files of the
> series between selecting them in the file open dialog and appearing of
> the Apply button.
>
> Anyway, I'll try to use again the mapping time step value -> file name
> and iron out the segfault on deletion of the reader object.
>
> I'll post the solution in case I can work it out and someone is
> interested.
>
> Karl
>
>
>
>
>
> **** Kenneth Moreland
> *** Sandia National Laboratories
> ***********
> *** *** *** email: kmorel at sandia.gov
> ** *** ** phone: (505) 844-8919
> *** web: http://www.cs.unm.edu/~kmorel
>
**** Kenneth Moreland
*** Sandia National Laboratories
***********
*** *** *** email: kmorel at sandia.gov
** *** ** phone: (505) 844-8919
*** web: http://www.cs.unm.edu/~kmorel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20090910/9744263d/attachment-0001.htm>
More information about the ParaView
mailing list