[Paraview] ParaView thinks that it does not have a wind blade reader

Tue Aug 28 14:42:50 EDT 2012

As far as the gradient filter is concerned: I pick the UVW array in the
GUI.  It does compute a gradient, but only for one array -- "A-Scale
turbulence" (all variables are loaded).  So picking a different array in
the GUI doesn't seem to make a difference as it keeps computing gradient
for that one input (point data) array.  That's why it can't compute
vorticity or q-criterion because a-scale turbulence is a 1-component array.

Might be because I'm using an older version of the repo from late July.

Sohail

On Tue, Aug 28, 2012 at 11:51 AM, Sohail Shafii <ssshafii at ucdavis.edu>wrote:

> Well I guess it's not a problem in the derived functions if I check to see
> if MPI is available; if not, call the superclass's version of the virtual
> function.
>
>
> On Tue, Aug 28, 2012 at 11:32 AM, Sohail Shafii <ssshafii at ucdavis.edu>wrote:
>
>> MPI is only used if it's available. It tries to see if there is a
>> MPIController available.  If not, then the number of ranks is 1, which
>> means that the parallel class calls the super class's requestdata (super
>> class is the serial wind blade reader).
>>
>> However, the parallel class overrides certain functions to make use of
>> MPI. So even the parallel instantiation will call the superclass's
>> requestdata, the overriden functions that are called from that point on
>> assume that MPI is available...this is a problem when the MPI controller is
>> not available.
>>
>> On another note, it appears that the rectilinear grid that I am reading
>> (VTK file, structured points I think) is read by the vtkFileSeriesReader,
>> which makes use of the vtkPDataSetReader class.  Strange that if I use
>> vtkPDataSetReader myself in C++ code that uses VTK and not paraview,
>> bounds/cell information is not generated. So I think that ParaView does
>> something else besides using that reader...hmmm.
>>
>>
>> On Mon, Aug 27, 2012 at 7:32 PM, Andy Bauer <andy.bauer at kitware.com>wrote:
>>
>>>
>>>
>>> On Mon, Aug 27, 2012 at 6:53 PM, Sohail Shafii <ssshafii at ucdavis.edu>wrote:
>>>
>>>> I see...so if MPI is running then it will create a parallel version of
>>>> the class. Otherwise, it will create the normal reader.
>>>>
>>>
>>> Technically, if ParaView is built with MPI enabled it will create the
>>> parallel version of the class every time whether or not the client, server,
>>> or python scripts are actually run with mpiexec, mpirun, or whatever is
>>> used to start the executable. So you can't rely on MPI being initialized
>>> and should not initialize it yourself in your class as that can cause
>>> problems as well. I think that's what you meant but wanted to be as clear
>>> as possible for the implications for the classes.
>>>
>>>
>>>>
>>>> As far as the check goes for a multiprocess controller; while it does
>>>> call the parent request data in the serial case, the inherited parallel
>>>> class overrides some of the methods.  Which is a problem because it refers
>>>> to its own file pointer in those methods (it uses MPI_File instead of a
>>>> FILE*). Hmm.
>>>>
>>>> Sohail
>>>>
>>>>
>>>> On Thu, Aug 23, 2012 at 3:20 PM, Andy Bauer <andy.bauer at kitware.com>wrote:
>>>>
>>>>> Hi Sohail,
>>>>>
>>>>> Sorry for the slow reply but I wanted to make sure how things should
>>>>> work first and that took a bit to figure out. Anyways, this is closer to
>>>>> what's needed but the way it is supposed to work in VTK is that we use
>>>>> vtkObjectFactory to determine which one should be created when
>>>>> vtkWindBladeReader::New() is called. This is done at build time based on
>>>>> whether or not VTK is built with MPI. So if VTK and ParaView are built in
>>>>> parallel then there is code added that creates a vtkPWindBladeReader when
>>>>> vtkWindBladeReader::New() is called. This is regardless of whether or not
>>>>> the executable is run with mpi initialized or not. Because of this, the
>>>>> parallel version of the reader is responsible for checking whether mpi is
>>>>> initialized, ideally by doing
>>>>> vtkMultiProcessController::GetGlobalController()->IsA("vtkMPIController")
>>>>> is true, and if it isn't it should just call the parent class's methods for
>>>>> reading in the files. You may want to look at vtkPNrrdReader and the
>>>>> CMakeLists.txt file in that directory to see how they did things. Note
>>>>> though that if it's a single process running that it doesn't do any MPI
>>>>> calls even in the parallel version so your code will be slightly different
>>>>> than theirs.
>>>>>
>>>>> As for other parts of the email below:
>>>>>
>>>>> 1) yes, please keep cc'ing the list as there are some details in here
>>>>> that will be helpful to other people trying to implement their own parallel
>>>>> readers or filters
>>>>>
>>>>> 2) I'm not sure what's going on with the gradient filter but am
>>>>> guessing that it has to do with the multiblock data set and that there
>>>>> probably isn't that same array in each block. But that's just a guess. Did
>>>>> you try extracting the "air" block and do the vorticity and/or q criterion
>>>>> computation on that?
>>>>>
>>>>> 3) vtkSMReaderFactory may be the class you're looking for to determine
>>>>> which reader is getting used. ParaViewReaders.xml has a list of readers
>>>>> that it will try to read in a file.
>>>>>
>>>>> Andy
>>>>>
>>>>>
>>>>> On Tue, Aug 21, 2012 at 7:11 PM, Sohail Shafii <ssshafii at ucdavis.edu>wrote:
>>>>>
>>>>>>  <sent via another e-mail address; yahoo prevents me from uploading
>>>>>> stuff because it's stupid>
>>>>>>
>>>>>> Here's the first version.  Do you want me to keep ccing the list? Not
>>>>>> sure if that's necessary at this point because this is related to the
>>>>>> development of ParaView and not help.
>>>>>>
>>>>>> I've run it with an older version of the master, and it seems to read
>>>>>> in a test .wind file just fine.  For now I had to store the
>>>>>> vtkPWindBladeReader* files into VTK/Parallel/MPI since I don't have
>>>>>> VTK/IO/MPIParallel module yet.  Now when I run the gradient filter after
>>>>>> the field, the gradient filter thinks that the input array does not have
>>>>>> the appropriate number of components for Q criterion or vorticity...but it
>>>>>> should (UVW does have three components per tuple).  Might be my outdated
>>>>>> copy of the git repo.  Haven't debugged that gradient code yet.
>>>>>>
>>>>>> *I was wondering if this is the type of code that you want?  Or
>>>>>> should the classes be designed differently?*  Also, I have only run
>>>>>> it parallel because if I try to instantiate it inside of the python shell
>>>>>> in serial it tries to run RequestInformation (which assumes that the file
>>>>>> path exists but does not).  It's a work-in-progress but it's a start.
>>>>>>
>>>>>> A random question: if ParaView reads a file, where would I put a
>>>>>> breakpoint that tells me which reader it ends up picking??  I have some
>>>>>> separate VTK code that reads in a rectilinear grid, but does not generate
>>>>>> cell/bounds information. ParaView does generate information about the
>>>>>> bounds though.
>>>>>>
>>>>>> Sohail
>>>>>>
>>>>>>   ------------------------------
>>>>>> *From:* David E DeMarle <dave.demarle at kitware.com>
>>>>>> *To:* Sohail Shafii <sohailshafii at yahoo.com>
>>>>>> *Cc:* Andy Bauer <andy.bauer at kitware.com>; "migichen at gmail.com" <
>>>>>> migichen at gmail.com>; "paraview at paraview.org" <paraview at paraview.org>;
>>>>>> "Woodring, Jonathan L" <woodring at lanl.gov>
>>>>>> *Sent:* Friday, July 20, 2012 10:02 AM
>>>>>> *Subject:* Re: [Paraview] ParaView thinks that it does not have a
>>>>>> wind blade reader
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> In today's git master it now sits in
>>>>>> VTK/IO/Parallel/vtkWindbladeReader.
>>>>>>
>>>>>> The serial version should go in VTK/IO/Geometry.
>>>>>>
>>>>>> The MPI version should go in a new module VTK/IO/MPIParallel.
>>>>>> (VTK/IO/Parallel depends only on vtkSocketController now and should
>>>>>> not depend on vtkMPIController).
>>>>>>
>>>>>> Finally when that is done we can remove the flag in paraview's xml
>>>>>> that prevents it from being used in serial.
>>>>>>
>>>>>> David E DeMarle
>>>>>> Kitware, Inc.
>>>>>> R&D Engineer
>>>>>> 21 Corporate Drive
>>>>>> Clifton Park, NY 12065-8662
>>>>>> Phone: 518-881-4909
>>>>>>
>>>>>>
>>>>>> On Thu, Jul 19, 2012 at 7:08 PM, Sohail Shafii <
>>>>>> sohailshafii at yahoo.com> wrote:
>>>>>> > Well it seems like #1 and #2 can be solved by writing a new
>>>>>> parallel wind
>>>>>> > blade reader class. I had similar experience by creating distributed
>>>>>> > versions of class that I created in other vtk directories. The
>>>>>> normal wind
>>>>>> > blade reader would have no parallel calls, which would address #1.
>>>>>> >
>>>>>> > What I find interesting is that in your directories, the wind blade
>>>>>> reader
>>>>>> > is located in VTK/IO/Parallel. For the ParaView snapshot that I
>>>>>> downloaded,
>>>>>> > as well as the 3.14 source, it's still in VTK/Parallel.  No
>>>>>> Parallel folder
>>>>>> > in VTK/IO. Is this a recent change?
>>>>>> >
>>>>>> > So maybe the ideal folder layout would be:
>>>>>> >
>>>>>> > VTK/IO/WindBladeReader.*
>>>>>> >
>>>>>> > VTK/IO/Parallel/DistributedWindBladeReader.*
>>>>>> >
>>>>>> > I can make changes to the WindBladeReader to remove the MPI-IO
>>>>>> calls and
>>>>>> > make a distributed wind blade reader class in case you want to add
>>>>>> them to
>>>>>> > the repository. The one template I have for the parallel/serial
>>>>>> split is the
>>>>>> > vtkStreamTracer and its distributed counterparts so I can use those
>>>>>> as a
>>>>>> > guide.  If you have better recommendations please let me know.
>>>>>> >
>>>>>> > Sohail
>>>>>> >
>>>>>> > ________________________________
>>>>>> > From: David E DeMarle <dave.demarle at kitware.com>
>>>>>> > To: Sohail Shafii <sohailshafii at yahoo.com>
>>>>>> > Cc: Andy Bauer <andy.bauer at kitware.com>; "migichen at gmail.com"
>>>>>> > <migichen at gmail.com>; "paraview at paraview.org" <
>>>>>> paraview at paraview.org>;
>>>>>> > "Woodring, Jonathan L" <woodring at lanl.gov>
>>>>>> > Sent: Thursday, July 19, 2012 3:20 PM
>>>>>> >
>>>>>> > Subject: Re: [Paraview] ParaView thinks that it does not have a
>>>>>> wind blade
>>>>>> > reader
>>>>>> >
>>>>>> > Two issues need to be fixed.
>>>>>> > 1) that behavior in 3.14 is there because of this fix which we had
>>>>>> to
>>>>>> > rush in just before the 3.14 release.
>>>>>> >
>>>>>> > Committer: Utkarsh Ayachit <utkarsh.ayachit at kitware.com>
>>>>>> 2012-02-08
>>>>>> > 17:12:18
>>>>>> > Parent: e401d569f6e54c49c3497852463f6b13a01da8b6 (Update version
>>>>>> after
>>>>>> > 3.14.0-RC1 tag.)
>>>>>> > Child:  99fdb1aeb5997c29b43393f45a478c4b53a23109 (Merge topic
>>>>>> > 'fix_windblade_segfault')
>>>>>> > Branches: many (31)
>>>>>> > Follows: v3.14.0-RC1
>>>>>> > Precedes: v3.14.0-RC2
>>>>>> >
>>>>>> >    Fix segfault with ParaView built with MPI when opening *.vtr
>>>>>> files.
>>>>>> >
>>>>>> >    The problem was that vtkWindBlade reader was getting created to
>>>>>> test
>>>>>> >    "CanReadFile" and that would cause segfault unless MPI was
>>>>>> initialized.
>>>>>> >
>>>>>> >    Solved the problem by added support to vtkSMReaderFactory to
>>>>>> respect the
>>>>>> >    ProcessSupport flag on SourceProxy and change windblade reader
>>>>>> XML
>>>>>> > to say that
>>>>>> >    it only works in parallel.
>>>>>> >
>>>>>> > In other words, Windblade reader's Constructor should not take it
>>>>>> upon
>>>>>> > itself to initialize MPI.
>>>>>> >
>>>>>> > Once that is fixed, we get to issue 2 before paraview 4.0
>>>>>> >
>>>>>> > 2) VTK's new modularization has the rule that no module can behave
>>>>>> > differently with parallel (or most other cmake config options)
>>>>>> turned
>>>>>> > configured on or off. This is so because it makes it very hard to
>>>>>> > package VTK when the libs behave differently in different
>>>>>> situations.
>>>>>> >
>>>>>> > Ie, we can't have any more
>>>>>> > #ifdef VTK_USE_MPI
>>>>>> > do something
>>>>>> > #else
>>>>>> > do something different
>>>>>> > #endif
>>>>>> > in VTK classes.
>>>>>> >
>>>>>> > Now if we want a class to behave differently when MPI is turned on
>>>>>> or
>>>>>> > off we have to make a serial only version in a non MPI dependent
>>>>>> > module and a subclass of that in an MPI dependent module.
>>>>>> >
>>>>>> > David E DeMarle
>>>>>> > Kitware, Inc.
>>>>>> > R&D Engineer
>>>>>> > 21 Corporate Drive
>>>>>> > Clifton Park, NY 12065-8662
>>>>>> > Phone: 518-881-4909
>>>>>> >
>>>>>> >
>>>>>> > On Thu, Jul 19, 2012 at 3:52 PM, Sohail Shafii <
>>>>>> sohailshafii at yahoo.com>
>>>>>> > wrote:
>>>>>> >> When running serially, vtkSMReaderFactory::CanReadFile sees that
>>>>>> the
>>>>>> >> session
>>>>>> >> has a single process.  As such, on line 660 it realizes that the
>>>>>> source
>>>>>> >> proxy's process support (in this case the source is a
>>>>>> vtkWindBladeReader
>>>>>> >> proxy) is multiple processes, which causes a return of "false"
>>>>>> from that
>>>>>> >> function.  This makes ParaView think that the wind blade reader
>>>>>> can't read
>>>>>> >> a
>>>>>> >> .wind file in serial.  According to Andy, the reader works find in
>>>>>> >> parallel.
>>>>>> >>
>>>>>> >> I assume that a fix for this is pretty trivial? It never did this
>>>>>> before
>>>>>> >> in
>>>>>> >> previous versions of ParaView and I think that the wind blade
>>>>>> reader code
>>>>>> >> has remained mostly the same. Andy mentioned some details in
>>>>>> person when
>>>>>> >> he
>>>>>> >> was talking with Jon but at that time I didn't know as much as I
>>>>>> do now.
>>>>>> >> There is a mention below about enabling the filter in serial and
>>>>>> parallel
>>>>>> >> --
>>>>>> >> is this done in a xml file?
>>>>>> >>
>>>>>> >> Sohail
>>>>>> >>
>>>>>> >> ________________________________
>>>>>> >> From: David E DeMarle <dave.demarle at kitware.com>
>>>>>> >> To: Sohail Shafii <sohailshafii at yahoo.com>
>>>>>> >> Cc: Andy Bauer <andy.bauer at kitware.com>; "migichen at gmail.com"
>>>>>> >> <migichen at gmail.com>; "paraview at paraview.org" <
>>>>>> paraview at paraview.org>;
>>>>>> >> "Woodring, Jonathan L" <woodring at lanl.gov>
>>>>>> >> Sent: Tuesday, July 17, 2012 7:00 PM
>>>>>> >>
>>>>>> >> Subject: Re: [Paraview] ParaView thinks that it does not have a
>>>>>> wind blade
>>>>>> >> reader
>>>>>> >>
>>>>>> >> Each file that has an association with that file extension, made
>>>>>> via the
>>>>>> >> XML, has its CanReadFile method called. Put a breakpoint there to
>>>>>> >> determine
>>>>>> >> if ParaView even knows about that reader.
>>>>>> >>
>>>>>> >> On Tuesday, July 17, 2012, Sohail Shafii wrote:
>>>>>> >>
>>>>>> >> Sorry, but I meant to ask a different question.  Here's what Andy
>>>>>> said
>>>>>> >> before, which relates to my concern:
>>>>>> >> ">
>>>>>> >>> I went through the code and it jogged my memory about previous
>>>>>> problems
>>>>>> >>> with
>>>>>> >>> it. It had an explicit MPI call in the constructor that would
>>>>>> cause
>>>>>> >>> problems
>>>>>> >>> when paraview wasn't getting built with MPI. Any time a user
>>>>>> tried to
>>>>>> >>> open
>>>>>> >>> an unknown file it would try to create a windblade reader to see
>>>>>> if the
>>>>>> >>> reader could handle the file and this would cause a crash. If you
>>>>>> launch
>>>>>> >>> a
>>>>>> >>> separate pvserver the wind blade reader will be available in the
>>>>>> gui."
>>>>>> >>
>>>>>> >> What I meant was; which ParaView class "creates" or "tests" a
>>>>>> reader to
>>>>>> >> see
>>>>>> >> if it can handle a file? The test code you specified has a
>>>>>> predefined data
>>>>>> >> set that is loaded in, and I think is more of an example to your
>>>>>> average
>>>>>> >> user. Correct me if I'm wrong.
>>>>>> >>
>>>>>> >> So basically ParaView will recognize the file extension, tries to
>>>>>> load it,
>>>>>> >> crashes somewhere, then asks you specify the data format from a
>>>>>> dialog.
>>>>>> >> I'm
>>>>>> >> trying to figure out where it crashes so that I can see how that
>>>>>> relates
>>>>>> >> to
>>>>>> >> the wind blade reader class and any MPI calls that are in there.
>>>>>> >>
>>>>>> >> Sohail
>>>>>> >>
>>>>>> >> Subject: Re: [Paraview] ParaView thinks that it does not have a
>>>>>> wind blade
>>>>>> >> reader
>>>>>> >>
>>>>>> >> kargad:ParaView demarle$ find . -name "*WindBlade*" -print
>>>>>> >> ---> ./VTK/IO/Parallel/Testing/Cxx/TestWindBladeReader.cxx
>>>>>> >> ./VTK/IO/Parallel/vtkWindBladeReader.cxx
>>>>>> >> ./VTK/IO/Parallel/vtkWindBladeReader.h
>>>>>> >>
>>>>>> >> David E DeMarle
>>>>>> >> Kitware, Inc.
>>>>>> >> R&D Engineer
>>>>>> >> 21 Corporate Drive
>>>>>> >> Clifton Park, NY 12065-8662
>>>>>> >> Phone: 518-881-4909
>>>>>> >>
>>>>>> >>
>>>>>> >> On Tue, Jul 17, 2012 at 2:09 PM, Sohail Shafii <
>>>>>> sohailshafii at yahoo.com>
>>>>>> >> wrote:
>>>>>> >>> What is the class that is responsible for testing to reader to
>>>>>> see if it
>>>>>> >>> can
>>>>>> >>> open the .wind file?
>>>>>> >>>
>>>>>> >>> Sohail
>>>>>> >>>
>>>>>> >>> ________________________________
>>>>>> >>> From: Andy Bauer <andy.bauer at kitware.com>
>>>>>> >>> To: Sohail Shafii <sohailshafii at yahoo.com>
>>>>>> >>> Cc: "Woodring, Jonathan L" <woodring at lanl.gov>; Sebastien
>>>>>> Jourdain
>>>>>> >>> <sebastien.jourdain at kitware.com>; "migichen at gmail.com"
>>>>>> >>> <migichen at gmail.com>;
>>>>>> >>> "paraview at paraview.org" <paraview at paraview.org>
>>>>>> >>> Sent: Wednesday, July 11, 2012 10:53 AM
>>>>>> >>>
>>>>>> >>> Subject: Re: [Paraview] ParaView thinks that it does not have a
>>>>>> wind
>>>>>> >>> blade
>>>>>> >>> reader
>>>>>> >>>
>>>>>> >>> I was reluctant to enable the reader in serial (i.e. not using
>>>>>> MPI IO
>>>>>> >>> which
>>>>>> >>> I think is getting done by default with the parallel build)
>>>>>> because of
>>>>>> >>> the
>>>>>> >>> following warnings that I got:
>>>>>> >>>
>>>>>> >>> Warning: In /home/acbauer/CODE/ParaView/
>>>>>> >>> ParaView/VTK/IO/Parallel/vtkWindBladeReader.cxx, line 942
>>>>>> >>> vtkWindBladeReader (0x3a0c800): WindBladeReader error reading
>>>>>> file:
>>>>>> >>>
>>>>>> /home/acbauer/DATA/VTKLargeData/Data/WindBladeReader/test1_topo.wind
>>>>>> >>> Premature EOF while reading block of data. Expected 896000 but
>>>>>> got 0
>>>>>> >>>
>>>>>> >>> I just assumed that it was getting bad results so I didn't check
>>>>>> the
>>>>>> >>> serial
>>>>>> >>> output with the parallel output. In either case the code should be
>>>>>> >>> changed
>>>>>> >>> before enabling it in serial so that it reads the files correctly
>>>>>> and
>>>>>> >>> doesn't produce any warning messages.
>>>>>> >>>
>>>>>> >>> Andy
>>>>>> >>>
>>>>>> >>> On Wed, Jul 11, 2012 at 12:04 PM, Sohail Shafii <
>>>>>> sohailshafii at yahoo.com>
>>>>>> >>> wrote:
>>>>>> >>>
>>>>>> >>> I'm not aware of any option to build/link against mpi-io.  This
>>>>>> is just a
>>>>>> >>> standard build with mpi support that suffers from this issue.
>>>>>> >>>
>>>>>> >>> The ParaView website's builds also have the same bug.
>>>>>> >>>
>>>>>> >>> ________________________________
>>>>>> >>> From: "Woodring, Jonathan L" <woodring at lanl.gov>
>>>>>> >>> To: Sohail Shafii <sohailshafii at yahoo.com>; Andy Bauer
>>>>>> >>> <andy.bauer at kitware.com>; Sebastien Jourdain
>>>>>> >>> <sebastien.jourdain at kitware.com>
>>>>>> >>> Cc: "migichen at gmail.com" <migichen at gmail.com>; "
>>>>>> paraview at paraview.org"
>>>>>> >>> <paraview at paraview.org>
>>>>>> >>> Sent: Tuesday, July 10, 2012 10:06 AM
>>>>>> >>>
>>>>>> >>> Subject: Re: [Paraview] ParaView thinks that it does not have a
>>>>>> wind
>>>>>> >>> blade
>>>>>> >>> reader
>>>>>> >>>
>>>>>> >>> I added the MPI calls to be in sync with what was being added in
>>>>>> the
>>>>>> >>> simulation.
>>>>>> >>>
>>>>>> >>> In the SuperComputing demo last year, we updated both the HiGrad
>>>>>> FireTec
>>>>>> >>> to
>>>>>> >>> use MPI-IO and likewise ParaView to do the same thing.
>>>>>> >>>
>>>>>> >>> It w
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >> --
>>>>>> >> David E DeMarle
>>>>>> >> Kitware, Inc.
>>>>>> >> R&D Engineer
>>>>>> >> 21 Corporate Drive
>>>>>> >> Clifton Park, NY 12065-8662
>>>>>> >> Phone: 518-881-4909
>>>>>> >>
>>>>>> >>
>>>>>> >
>>>>>> >
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20120828/9b1284d5/attachment-0001.htm>