[Paraview] ParaView thinks that it does not have a wind blade reader

Mon Aug 27 18:53:00 EDT 2012

I see...so if MPI is running then it will create a parallel version of the
class. Otherwise, it will create the normal reader.

As far as the check goes for a multiprocess controller; while it does call
the parent request data in the serial case, the inherited parallel class
overrides some of the methods.  Which is a problem because it refers to its
own file pointer in those methods (it uses MPI_File instead of a FILE*).
Hmm.

Sohail

On Thu, Aug 23, 2012 at 3:20 PM, Andy Bauer <andy.bauer at kitware.com> wrote:

> Hi Sohail,
>
> Sorry for the slow reply but I wanted to make sure how things should work
> first and that took a bit to figure out. Anyways, this is closer to what's
> needed but the way it is supposed to work in VTK is that we use
> vtkObjectFactory to determine which one should be created when
> vtkWindBladeReader::New() is called. This is done at build time based on
> whether or not VTK is built with MPI. So if VTK and ParaView are built in
> parallel then there is code added that creates a vtkPWindBladeReader when
> vtkWindBladeReader::New() is called. This is regardless of whether or not
> the executable is run with mpi initialized or not. Because of this, the
> parallel version of the reader is responsible for checking whether mpi is
> initialized, ideally by doing
> vtkMultiProcessController::GetGlobalController()->IsA("vtkMPIController")
> is true, and if it isn't it should just call the parent class's methods for
> reading in the files. You may want to look at vtkPNrrdReader and the
> CMakeLists.txt file in that directory to see how they did things. Note
> though that if it's a single process running that it doesn't do any MPI
> calls even in the parallel version so your code will be slightly different
> than theirs.
>
> As for other parts of the email below:
>
> 1) yes, please keep cc'ing the list as there are some details in here that
> will be helpful to other people trying to implement their own parallel
> readers or filters
>
> 2) I'm not sure what's going on with the gradient filter but am guessing
> that it has to do with the multiblock data set and that there probably
> isn't that same array in each block. But that's just a guess. Did you try
> extracting the "air" block and do the vorticity and/or q criterion
> computation on that?
>
> 3) vtkSMReaderFactory may be the class you're looking for to determine
> which reader is getting used. ParaViewReaders.xml has a list of readers
> that it will try to read in a file.
>
> Andy
>
>
> On Tue, Aug 21, 2012 at 7:11 PM, Sohail Shafii <ssshafii at ucdavis.edu>wrote:
>
>>  <sent via another e-mail address; yahoo prevents me from uploading
>> stuff because it's stupid>
>>
>> Here's the first version.  Do you want me to keep ccing the list? Not
>> sure if that's necessary at this point because this is related to the
>> development of ParaView and not help.
>>
>> I've run it with an older version of the master, and it seems to read in
>> a test .wind file just fine.  For now I had to store the
>> vtkPWindBladeReader* files into VTK/Parallel/MPI since I don't have
>> VTK/IO/MPIParallel module yet.  Now when I run the gradient filter after
>> the field, the gradient filter thinks that the input array does not have
>> the appropriate number of components for Q criterion or vorticity...but it
>> should (UVW does have three components per tuple).  Might be my outdated
>> copy of the git repo.  Haven't debugged that gradient code yet.
>>
>> *I was wondering if this is the type of code that you want?  Or should
>> the classes be designed differently?*  Also, I have only run it parallel
>> because if I try to instantiate it inside of the python shell in serial it
>> tries to run RequestInformation (which assumes that the file path exists
>> but does not).  It's a work-in-progress but it's a start.
>>
>> A random question: if ParaView reads a file, where would I put a
>> breakpoint that tells me which reader it ends up picking??  I have some
>> separate VTK code that reads in a rectilinear grid, but does not generate
>> cell/bounds information. ParaView does generate information about the
>> bounds though.
>>
>> Sohail
>>
>>   ------------------------------
>> *From:* David E DeMarle <dave.demarle at kitware.com>
>> *To:* Sohail Shafii <sohailshafii at yahoo.com>
>> *Cc:* Andy Bauer <andy.bauer at kitware.com>; "migichen at gmail.com" <
>> migichen at gmail.com>; "paraview at paraview.org" <paraview at paraview.org>;
>> "Woodring, Jonathan L" <woodring at lanl.gov>
>> *Sent:* Friday, July 20, 2012 10:02 AM
>> *Subject:* Re: [Paraview] ParaView thinks that it does not have a wind
>> blade reader
>>
>> Thanks.
>>
>> In today's git master it now sits in VTK/IO/Parallel/vtkWindbladeReader.
>>
>> The serial version should go in VTK/IO/Geometry.
>>
>> The MPI version should go in a new module VTK/IO/MPIParallel.
>> (VTK/IO/Parallel depends only on vtkSocketController now and should
>> not depend on vtkMPIController).
>>
>> Finally when that is done we can remove the flag in paraview's xml
>> that prevents it from being used in serial.
>>
>> David E DeMarle
>> Kitware, Inc.
>> R&D Engineer
>> 21 Corporate Drive
>> Clifton Park, NY 12065-8662
>> Phone: 518-881-4909
>>
>>
>> On Thu, Jul 19, 2012 at 7:08 PM, Sohail Shafii <sohailshafii at yahoo.com>
>> wrote:
>> > Well it seems like #1 and #2 can be solved by writing a new parallel
>> wind
>> > blade reader class. I had similar experience by creating distributed
>> > versions of class that I created in other vtk directories. The normal
>> wind
>> > blade reader would have no parallel calls, which would address #1.
>> >
>> > What I find interesting is that in your directories, the wind blade
>> reader
>> > is located in VTK/IO/Parallel. For the ParaView snapshot that I
>> downloaded,
>> > as well as the 3.14 source, it's still in VTK/Parallel.  No Parallel
>> folder
>> > in VTK/IO. Is this a recent change?
>> >
>> > So maybe the ideal folder layout would be:
>> >
>> > VTK/IO/WindBladeReader.*
>> >
>> > VTK/IO/Parallel/DistributedWindBladeReader.*
>> >
>> > I can make changes to the WindBladeReader to remove the MPI-IO calls and
>> > make a distributed wind blade reader class in case you want to add them
>> to
>> > the repository. The one template I have for the parallel/serial split
>> is the
>> > vtkStreamTracer and its distributed counterparts so I can use those as a
>> > guide.  If you have better recommendations please let me know.
>> >
>> > Sohail
>> >
>> > ________________________________
>> > From: David E DeMarle <dave.demarle at kitware.com>
>> > To: Sohail Shafii <sohailshafii at yahoo.com>
>> > Cc: Andy Bauer <andy.bauer at kitware.com>; "migichen at gmail.com"
>> > <migichen at gmail.com>; "paraview at paraview.org" <paraview at paraview.org>;
>> > "Woodring, Jonathan L" <woodring at lanl.gov>
>> > Sent: Thursday, July 19, 2012 3:20 PM
>> >
>> > Subject: Re: [Paraview] ParaView thinks that it does not have a wind
>> blade
>> > reader
>> >
>> > Two issues need to be fixed.
>> > 1) that behavior in 3.14 is there because of this fix which we had to
>> > rush in just before the 3.14 release.
>> >
>> > Committer: Utkarsh Ayachit <utkarsh.ayachit at kitware.com>  2012-02-08
>> > 17:12:18
>> > Parent: e401d569f6e54c49c3497852463f6b13a01da8b6 (Update version after
>> > 3.14.0-RC1 tag.)
>> > Child:  99fdb1aeb5997c29b43393f45a478c4b53a23109 (Merge topic
>> > 'fix_windblade_segfault')
>> > Branches: many (31)
>> > Follows: v3.14.0-RC1
>> > Precedes: v3.14.0-RC2
>> >
>> >    Fix segfault with ParaView built with MPI when opening *.vtr files.
>> >
>> >    The problem was that vtkWindBlade reader was getting created to test
>> >    "CanReadFile" and that would cause segfault unless MPI was
>> initialized.
>> >
>> >    Solved the problem by added support to vtkSMReaderFactory to respect
>> the
>> >    ProcessSupport flag on SourceProxy and change windblade reader XML
>> > to say that
>> >    it only works in parallel.
>> >
>> > In other words, Windblade reader's Constructor should not take it upon
>> > itself to initialize MPI.
>> >
>> > Once that is fixed, we get to issue 2 before paraview 4.0
>> >
>> > 2) VTK's new modularization has the rule that no module can behave
>> > differently with parallel (or most other cmake config options) turned
>> > configured on or off. This is so because it makes it very hard to
>> > package VTK when the libs behave differently in different situations.
>> >
>> > Ie, we can't have any more
>> > #ifdef VTK_USE_MPI
>> > do something
>> > #else
>> > do something different
>> > #endif
>> > in VTK classes.
>> >
>> > Now if we want a class to behave differently when MPI is turned on or
>> > off we have to make a serial only version in a non MPI dependent
>> > module and a subclass of that in an MPI dependent module.
>> >
>> > David E DeMarle
>> > Kitware, Inc.
>> > R&D Engineer
>> > 21 Corporate Drive
>> > Clifton Park, NY 12065-8662
>> > Phone: 518-881-4909
>> >
>> >
>> > On Thu, Jul 19, 2012 at 3:52 PM, Sohail Shafii <sohailshafii at yahoo.com>
>> > wrote:
>> >> When running serially, vtkSMReaderFactory::CanReadFile sees that the
>> >> session
>> >> has a single process.  As such, on line 660 it realizes that the source
>> >> proxy's process support (in this case the source is a
>> vtkWindBladeReader
>> >> proxy) is multiple processes, which causes a return of "false" from
>> that
>> >> function.  This makes ParaView think that the wind blade reader can't
>> read
>> >> a
>> >> .wind file in serial.  According to Andy, the reader works find in
>> >> parallel.
>> >>
>> >> I assume that a fix for this is pretty trivial? It never did this
>> before
>> >> in
>> >> previous versions of ParaView and I think that the wind blade reader
>> code
>> >> has remained mostly the same. Andy mentioned some details in person
>> when
>> >> he
>> >> was talking with Jon but at that time I didn't know as much as I do
>> now.
>> >> There is a mention below about enabling the filter in serial and
>> parallel
>> >> --
>> >> is this done in a xml file?
>> >>
>> >> Sohail
>> >>
>> >> ________________________________
>> >> From: David E DeMarle <dave.demarle at kitware.com>
>> >> To: Sohail Shafii <sohailshafii at yahoo.com>
>> >> Cc: Andy Bauer <andy.bauer at kitware.com>; "migichen at gmail.com"
>> >> <migichen at gmail.com>; "paraview at paraview.org" <paraview at paraview.org>;
>> >> "Woodring, Jonathan L" <woodring at lanl.gov>
>> >> Sent: Tuesday, July 17, 2012 7:00 PM
>> >>
>> >> Subject: Re: [Paraview] ParaView thinks that it does not have a wind
>> blade
>> >> reader
>> >>
>> >> Each file that has an association with that file extension, made via
>> the
>> >> XML, has its CanReadFile method called. Put a breakpoint there to
>> >> determine
>> >> if ParaView even knows about that reader.
>> >>
>> >> On Tuesday, July 17, 2012, Sohail Shafii wrote:
>> >>
>> >> Sorry, but I meant to ask a different question.  Here's what Andy said
>> >> before, which relates to my concern:
>> >> ">
>> >>> I went through the code and it jogged my memory about previous
>> problems
>> >>> with
>> >>> it. It had an explicit MPI call in the constructor that would cause
>> >>> problems
>> >>> when paraview wasn't getting built with MPI. Any time a user tried to
>> >>> open
>> >>> an unknown file it would try to create a windblade reader to see if
>> the
>> >>> reader could handle the file and this would cause a crash. If you
>> launch
>> >>> a
>> >>> separate pvserver the wind blade reader will be available in the gui."
>> >>
>> >> What I meant was; which ParaView class "creates" or "tests" a reader to
>> >> see
>> >> if it can handle a file? The test code you specified has a predefined
>> data
>> >> set that is loaded in, and I think is more of an example to your
>> average
>> >> user. Correct me if I'm wrong.
>> >>
>> >> So basically ParaView will recognize the file extension, tries to load
>> it,
>> >> crashes somewhere, then asks you specify the data format from a dialog.
>> >> I'm
>> >> trying to figure out where it crashes so that I can see how that
>> relates
>> >> to
>> >> the wind blade reader class and any MPI calls that are in there.
>> >>
>> >> Sohail
>> >>
>> >> Subject: Re: [Paraview] ParaView thinks that it does not have a wind
>> blade
>> >> reader
>> >>
>> >> kargad:ParaView demarle$ find . -name "*WindBlade*" -print
>> >> ---> ./VTK/IO/Parallel/Testing/Cxx/TestWindBladeReader.cxx
>> >> ./VTK/IO/Parallel/vtkWindBladeReader.cxx
>> >> ./VTK/IO/Parallel/vtkWindBladeReader.h
>> >>
>> >> David E DeMarle
>> >> Kitware, Inc.
>> >> R&D Engineer
>> >> 21 Corporate Drive
>> >> Clifton Park, NY 12065-8662
>> >> Phone: 518-881-4909
>> >>
>> >>
>> >> On Tue, Jul 17, 2012 at 2:09 PM, Sohail Shafii <sohailshafii at yahoo.com
>> >
>> >> wrote:
>> >>> What is the class that is responsible for testing to reader to see if
>> it
>> >>> can
>> >>> open the .wind file?
>> >>>
>> >>> Sohail
>> >>>
>> >>> ________________________________
>> >>> From: Andy Bauer <andy.bauer at kitware.com>
>> >>> To: Sohail Shafii <sohailshafii at yahoo.com>
>> >>> Cc: "Woodring, Jonathan L" <woodring at lanl.gov>; Sebastien Jourdain
>> >>> <sebastien.jourdain at kitware.com>; "migichen at gmail.com"
>> >>> <migichen at gmail.com>;
>> >>> "paraview at paraview.org" <paraview at paraview.org>
>> >>> Sent: Wednesday, July 11, 2012 10:53 AM
>> >>>
>> >>> Subject: Re: [Paraview] ParaView thinks that it does not have a wind
>> >>> blade
>> >>> reader
>> >>>
>> >>> I was reluctant to enable the reader in serial (i.e. not using MPI IO
>> >>> which
>> >>> I think is getting done by default with the parallel build) because of
>> >>> the
>> >>> following warnings that I got:
>> >>>
>> >>> Warning: In /home/acbauer/CODE/ParaView/
>> >>> ParaView/VTK/IO/Parallel/vtkWindBladeReader.cxx, line 942
>> >>> vtkWindBladeReader (0x3a0c800): WindBladeReader error reading file:
>> >>> /home/acbauer/DATA/VTKLargeData/Data/WindBladeReader/test1_topo.wind
>> >>> Premature EOF while reading block of data. Expected 896000 but got 0
>> >>>
>> >>> I just assumed that it was getting bad results so I didn't check the
>> >>> serial
>> >>> output with the parallel output. In either case the code should be
>> >>> changed
>> >>> before enabling it in serial so that it reads the files correctly and
>> >>> doesn't produce any warning messages.
>> >>>
>> >>> Andy
>> >>>
>> >>> On Wed, Jul 11, 2012 at 12:04 PM, Sohail Shafii <
>> sohailshafii at yahoo.com>
>> >>> wrote:
>> >>>
>> >>> I'm not aware of any option to build/link against mpi-io.  This is
>> just a
>> >>> standard build with mpi support that suffers from this issue.
>> >>>
>> >>> The ParaView website's builds also have the same bug.
>> >>>
>> >>> ________________________________
>> >>> From: "Woodring, Jonathan L" <woodring at lanl.gov>
>> >>> To: Sohail Shafii <sohailshafii at yahoo.com>; Andy Bauer
>> >>> <andy.bauer at kitware.com>; Sebastien Jourdain
>> >>> <sebastien.jourdain at kitware.com>
>> >>> Cc: "migichen at gmail.com" <migichen at gmail.com>; "paraview at paraview.org
>> "
>> >>> <paraview at paraview.org>
>> >>> Sent: Tuesday, July 10, 2012 10:06 AM
>> >>>
>> >>> Subject: Re: [Paraview] ParaView thinks that it does not have a wind
>> >>> blade
>> >>> reader
>> >>>
>> >>> I added the MPI calls to be in sync with what was being added in the
>> >>> simulation.
>> >>>
>> >>> In the SuperComputing demo last year, we updated both the HiGrad
>> FireTec
>> >>> to
>> >>> use MPI-IO and likewise ParaView to do the same thing.
>> >>>
>> >>> It w
>> >>
>> >>
>> >>
>> >> --
>> >> David E DeMarle
>> >> Kitware, Inc.
>> >> R&D Engineer
>> >> 21 Corporate Drive
>> >> Clifton Park, NY 12065-8662
>> >> Phone: 518-881-4909
>> >>
>> >>
>> >
>> >
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20120827/9dc50bf4/attachment-0001.htm>