[Amr] yt inside paraview

[Matthew Turk]

Hi Jorge,

Here's what I came up with.  I tested it by opening an Enzo dataset,
loading a few fields, and then creating a programmable filter.  It
looks like it received everything.

http://paste.enzotools.org/show/1698/

This is exciting!  It actually gave me good output.  So I suppose the
next question is, how should we proceed?  Do you think it's time to
start cleaning up the yt interface, as it does seem a bit cumbersome?
Or should we try to, for instance, create new derived fields yt knows
how to make?

Thanks,

Matt

On Fri, Jun 10, 2011 at 1:48 PM, Matthew Turk <matthewturk at gmail.com> wrote:
> Hi Jorge,
>
> I'm still working on the parent/child relationship.  I've implemented
> a stopgap that will accept "None" an then reconstruct the values; I
> recognize this is inefficient and I will replace it.
>
> The other thing is that now if you supply an argument "io" to the
> StreamHandler, it will use that instead of the previous method for
> stream io handling.
>
> This object should subclass yt.utilities.io_handler.BaseIOHandler and
> should implement the following methods:
>
> _read_data_set(self, grid, field)
> _read_field_names(self, grid)
> _read_data_slice(self, grid, field, axis, coord) # coord is an int
>
> It should also define a _data_style string attribute; in the future,
> once we've settled this down, we'll wrap the paraview IO style
> directly in and make this a natively understood method.  It should
> also define a _read_exception, which is used to tell if there's been a
> problem and yt should try to generate a field instead of reading it
> from disk.
>
> I'm going try to wrap what you've written into this framework and load
> it up here.  I'll ping the list and update the examples on the wiki
> page when that's done.
>
> -Matt
>
> These are all in 48ee9a3a6e4a in the bitbucket repo.
>
> On Fri, Jun 10, 2011 at 12:59 PM, Jorge Poco <jorge.poco at kitware.com> wrote:
>> Hi Matt
>> I have written the StreamFieldHandler and StreamHandler to interact with
>> Paraview, but I have a couple of questions:
>>
>> parent_ids will be change by child masking, right?
>> In the function __getitem__ of StreamFieldHandler, could be changed to
>> receive 2 parameter instead of 1?, these parameters would be index_grid and
>> field_name,  and should return an numpy.array in 3D. In way it's done now
>> could work, but the array returned will be 1D, so, I think I could handle it
>> in __getitem__ and return the correct array.
>>
>> The ProgrammableFilter filter is
>> here http://paste.enzotools.org/show/1695/ I was using
>> the tests/DD0010/moving7_0010.hierarchy data set.
>> Regards,
>>
>> On Fri, Jun 10, 2011 at 3:21 PM, Matthew Turk <matthewturk at gmail.com> wrote:
>>>
>>> As a quick note, the values should all differ by single multiplicative
>>> factor for each field.
>>>
>>> On Fri, Jun 10, 2011 at 12:19 PM, Matthew Turk <matthewturk at gmail.com>
>>> wrote:
>>> > Hi Jorge,
>>> >
>>> > Yup, we apply the unit conversions to get them back to CGS.
>>> > Temperature field takes no conversions.
>>> >
>>> > -Matt
>>> >
>>> > On Fri, Jun 10, 2011 at 12:18 PM, Jorge Poco <jorge.poco at kitware.com>
>>> > wrote:
>>> >> HI Matt
>>> >> Another doubt: I'm using the tests/DD0010/moving7_0010 data set. I have
>>> >> tried to load it using YT and Paraview.
>>> >> The problem is that grid values are different in both tools except
>>> >> temperature field.  I'm wondering if you
>>> >> do any conversions in the grid values when you load the data set.
>>> >> The code I'm using in yt is:
>>> >>>> pf =  load('tests/DD0010/moving7_0010')
>>> >>>> pf.h.grids[0]['Density'].min()
>>> >>>> pf.h.grids[0]['Density'].max()
>>> >> Regards,
>>> >> On Fri, Jun 10, 2011 at 2:40 PM, George Zagaris
>>> >> <george.zagaris at kitware.com>
>>> >> wrote:
>>> >>>
>>> >>> That's great!
>>> >>>
>>> >>> Yes the vtkHierarchicalBoxDataSet has IBLANK information that
>>> >>> can be supplied directly to yt.
>>> >>>
>>> >>> Best,
>>> >>> George
>>> >>>
>>> >>> On Fri, Jun 10, 2011 at 2:37 PM, Matthew Turk <matthewturk at gmail.com>
>>> >>> wrote:
>>> >>> > Hi Jorge,
>>> >>> >
>>> >>> > Nope, it should not be mandatory, although it currently is.  It is
>>> >>> > currently only used to calculate child masking, which I believe we
>>> >>> > can
>>> >>> > also get from vtkHierarchicalBoxDataSet?  I'll make this change
>>> >>> > later
>>> >>> > today.
>>> >>> >
>>> >>> > -Matt
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> > On Fri, Jun 10, 2011 at 11:35 AM, Jorge Poco
>>> >>> > <jorge.poco at kitware.com>
>>> >>> > wrote:
>>> >>> >> HI Matt
>>> >>> >> Looks nice.
>>> >>> >> In the 'stream' frontends, when the stream.StreamHandler(...) is
>>> >>> >> create,
>>> >>> >> one parameter is the parent_ids right? It's really mandatory to
>>> >>> >> send
>>> >>> >> it?
>>> >>> >> because the vtkHierarchicalBoxDataSet doesn't save this
>>> >>> >> information,
>>> >>> >> but we
>>> >>> >> can compute the parent/child relations.
>>> >>> >> Regards,
>>> >>> >>
>>> >>> >> On Fri, Jun 10, 2011 at 2:19 PM, Matthew Turk
>>> >>> >> <matthewturk at gmail.com>
>>> >>> >> wrote:
>>> >>> >>>
>>> >>> >>> Hi Jorge,
>>> >>> >>>
>>> >>> >>> I thought this might be of interest to you; I've started keeping
>>> >>> >>> track
>>> >>> >>> of how to interface with external codes such as ParaView here:
>>> >>> >>>
>>> >>> >>> http://hg.enzotools.org/yt/wiki/InterfacingWithYT
>>> >>> >>>
>>> >>> >>> I'll be updating this as need be with information about the
>>> >>> >>> in-memory
>>> >>> >>> datasets and so on.
>>> >>> >>>
>>> >>> >>> Best,
>>> >>> >>>
>>> >>> >>> Matt
>>> >>> >>>
>>> >>> >>> On Fri, May 27, 2011 at 2:08 PM, Matthew Turk
>>> >>> >>> <matthewturk at gmail.com>
>>> >>> >>> wrote:
>>> >>> >>> > Hi all,
>>> >>> >>> >
>>> >>> >>> > I've created a new data frontend in yt entitled "Stream" (the
>>> >>> >>> > name
>>> >>> >>> > is
>>> >>> >>> > up for discussion, if someone has a better idea -- Charles
>>> >>> >>> > pointed
>>> >>> >>> > out
>>> >>> >>> > today that it might get confused with time series data.)  The
>>> >>> >>> > idea
>>> >>> >>> > behind this is that the user can use it to construct an
>>> >>> >>> > arbitrary
>>> >>> >>> > dataset in memory.  I've tested this by proxying an existing
>>> >>> >>> > Enzo
>>> >>> >>> > dataset:
>>> >>> >>> >
>>> >>> >>> >
>>> >>> >>> >
>>> >>> >>> >
>>> >>> >>> > http://matthewturk.bitbucket.org/html/ba3fd37b-842c-4641-b21e-2d3f5268eefe-stream_proxy-py.html
>>> >>> >>> >
>>> >>> >>> > (downloadable at the top level domain, or by "yt_lodgeit.py
>>> >>> >>> > --download=1672 if you have a yt stack installed.)
>>> >>> >>> >
>>> >>> >>> > You have to construct both a StreamHandler that knows how to
>>> >>> >>> > pass
>>> >>> >>> > around the grid geometry and which, although not instantiated
>>> >>> >>> > with
>>> >>> >>> > them, requires certain attributes to be set.  In addition, a
>>> >>> >>> > fields
>>> >>> >>> > handler has to be created.  This script executes to completion
>>> >>> >>> > and
>>> >>> >>> > passes its tests; projecting and slicing require slightly more
>>> >>> >>> > effort,
>>> >>> >>> > because I wanted to pose to the Kitware group how the data
>>> >>> >>> > should be
>>> >>> >>> > evaluated in terms of units -- should it be assumed to have CGS
>>> >>> >>> > units
>>> >>> >>> > already?  How should the domain size be handled -- in yt right
>>> >>> >>> > now
>>> >>> >>> > units are distinct from the code definitions, but I don't know
>>> >>> >>> > how
>>> >>> >>> > Paraview handles this distinction.  (If you want to try setting
>>> >>> >>> > up a
>>> >>> >>> > PlotCollection, you can, but you will have to set
>>> >>> >>> > spf.units["cm"] =
>>> >>> >>> > spf.units["unitary"] = 1.0).
>>> >>> >>> >
>>> >>> >>> > Hopefully this can be a start, and we can continue to iterate on
>>> >>> >>> > the
>>> >>> >>> > API necessary to load data in yt from that already loaded in
>>> >>> >>> > Paraview.
>>> >>> >>> >  This is in the tip of the development yt repository,
>>> >>> >>> > 6d4a5588a550.
>>> >>> >>> >
>>> >>> >>> > Best,
>>> >>> >>> >
>>> >>> >>> > Matt
>>> >>> >>> >
>>> >>> >>> > On Thu, May 26, 2011 at 3:54 PM, Jorge Poco
>>> >>> >>> > <jorge.poco at kitware.com>
>>> >>> >>> > wrote:
>>> >>> >>> >> In the  meantime I will be working to import yt's data
>>> >>> >>> >> structures
>>> >>> >>> >> inside
>>> >>> >>> >> vtkHierarchicalBoxDataset using ProgrammableSources inside
>>> >>> >>> >> Python.
>>> >>> >>> >> Regards,
>>> >>> >>> >>
>>> >>> >>> >> On Thu, May 26, 2011 at 3:43 PM, Jorge Poco
>>> >>> >>> >> <jorge.poco at kitware.com>
>>> >>> >>> >> wrote:
>>> >>> >>> >>>
>>> >>> >>> >>> Hi
>>> >>> >>> >>> YT has some function to access the arrays data in numpy.array
>>> >>> >>> >>> objects
>>> >>> >>> >>> (
>>> >>> >>> >>> pf.h.grids[0]['Density']), using them  we can instantiate a
>>> >>> >>> >>> vtkHierarchicalDataset to be process into paraview.
>>> >>> >>> >>> On Thu, May 26, 2011 at 2:53 PM, Berk Geveci
>>> >>> >>> >>> <berk.geveci at kitware.com>
>>> >>> >>> >>> wrote:
>>> >>> >>> >>>>
>>> >>> >>> >>>> Awesome.
>>> >>> >>> >>>>
>>> >>> >>> >>>> What about from the other side? If we wanted to pass data
>>> >>> >>> >>>> structures
>>> >>> >>> >>>> that yt loaded (from an Enzo file for example) to
>>> >>> >>> >>>> VTK/ParaView
>>> >>> >>> >>>> for
>>> >>> >>> >>>> filtering/rendering?
>>> >>> >>> >>>>
>>> >>> >>> >>>> -berk
>>> >>> >>> >>>>
>>> >>> >>> >>>> On Thu, May 26, 2011 at 2:04 PM, Matthew Turk
>>> >>> >>> >>>> <matthewturk at gmail.com>
>>> >>> >>> >>>> wrote:
>>> >>> >>> >>>> > Hi Berk and Jorge,
>>> >>> >>> >>>> >
>>> >>> >>> >>>> > Okay, this makes sense.
>>> >>> >>> >>>> >
>>> >>> >>> >>>> > I will write up a new code frontend for yt that will act as
>>> >>> >>> >>>> > a
>>> >>> >>> >>>> > generalized system; this will draw upon the current in situ
>>> >>> >>> >>>> > Enzo
>>> >>> >>> >>>> > frontend, and will be designed to be instantiated with a
>>> >>> >>> >>>> > list
>>> >>> >>> >>>> > of
>>> >>> >>> >>>> > arrays and a set of callback functions.  We can them
>>> >>> >>> >>>> > implement
>>> >>> >>> >>>> > the
>>> >>> >>> >>>> > paraview-specific code inside this, or perhaps inside a
>>> >>> >>> >>>> > ProgrammableFilter inside paraview itself.  I'll ping the
>>> >>> >>> >>>> > list
>>> >>> >>> >>>> > back
>>> >>> >>> >>>> > when I have something up and running.
>>> >>> >>> >>>> >
>>> >>> >>> >>>> > Best,
>>> >>> >>> >>>> >
>>> >>> >>> >>>> > Matt
>>> >>> >>> >>>> >
>>> >>> >>> >>>> > On Thu, May 26, 2011 at 9:15 AM, Berk Geveci
>>> >>> >>> >>>> > <berk.geveci at kitware.com>
>>> >>> >>> >>>> > wrote:
>>> >>> >>> >>>> >> In VTK, an AMR mesh is represented as a collection of
>>> >>> >>> >>>> >> vtkUniformGrid
>>> >>> >>> >>>> >> objects. Each vtkUniformGrid has a set of Fortran-ordered
>>> >>> >>> >>>> >> data
>>> >>> >>> >>>> >> arrays
>>> >>> >>> >>>> >> that represent the variables. One exception is how vectors
>>> >>> >>> >>>> >> are
>>> >>> >>> >>>> >> stored.
>>> >>> >>> >>>> >> They are stored in hybrid Fortran - C ordering, i.e.
>>> >>> >>> >>>> >> vector
>>> >>> >>> >>>> >> component
>>> >>> >>> >>>> >> increases fastest then i, then j, then k. It is
>>> >>> >>> >>>> >> straightforward to
>>> >>> >>> >>>> >> pass these to Python and back using NumPy arrays or buffer
>>> >>> >>> >>>> >> objects
>>> >>> >>> >>>> >> at
>>> >>> >>> >>>> >> the lowest level. So we can write Python code to bundle
>>> >>> >>> >>>> >> these
>>> >>> >>> >>>> >> arrays
>>> >>> >>> >>>> >> into any form that yt (and Enzo when we start doing
>>> >>> >>> >>>> >> in-situ)
>>> >>> >>> >>>> >> wants. We
>>> >>> >>> >>>> >> may have to deep-copy the vectors because of the issue I
>>> >>> >>> >>>> >> described.
>>> >>> >>> >>>> >>
>>> >>> >>> >>>> >> On Wed, May 25, 2011 at 7:59 PM, Matthew Turk
>>> >>> >>> >>>> >> <matthewturk at gmail.com>
>>> >>> >>> >>>> >> wrote:
>>> >>> >>> >>>> >>> Hi Jorge,
>>> >>> >>> >>>> >>>
>>> >>> >>> >>>> >>> This is really cool!
>>> >>> >>> >>>> >>>
>>> >>> >>> >>>> >>> I've been thinking about how to invert this process, and
>>> >>> >>> >>>> >>> I
>>> >>> >>> >>>> >>> think
>>> >>> >>> >>>> >>> we
>>> >>> >>> >>>> >>> could address it by creating a new hierarchy of objects,
>>> >>> >>> >>>> >>> similar
>>> >>> >>> >>>> >>> to
>>> >>> >>> >>>> >>> the EnzoStaticOutputInMemory family of objects, that
>>> >>> >>> >>>> >>> enables
>>> >>> >>> >>>> >>> a
>>> >>> >>> >>>> >>> bi-directional transfer of information.  All that's
>>> >>> >>> >>>> >>> really
>>> >>> >>> >>>> >>> necessary
>>> >>> >>> >>>> >>> to get data in is to describe the mesh, describe the
>>> >>> >>> >>>> >>> parameters
>>> >>> >>> >>>> >>> of
>>> >>> >>> >>>> >>> the
>>> >>> >>> >>>> >>> simulation, and then communicate to yt how to access
>>> >>> >>> >>>> >>> data.
>>> >>> >>> >>>> >>>
>>> >>> >>> >>>> >>> What do you think the best mechanism for passing data
>>> >>> >>> >>>> >>> back
>>> >>> >>> >>>> >>> and
>>> >>> >>> >>>> >>> forth
>>> >>> >>> >>>> >>> is?  For in situ viz with Enzo, usually we do this by
>>> >>> >>> >>>> >>> constructing a
>>> >>> >>> >>>> >>> new module object in the C code, and then insert into
>>> >>> >>> >>>> >>> that a
>>> >>> >>> >>>> >>> set
>>> >>> >>> >>>> >>> of
>>> >>> >>> >>>> >>> arrays describing the mesh and a dictionary of
>>> >>> >>> >>>> >>> dictionaries
>>> >>> >>> >>>> >>> of
>>> >>> >>> >>>> >>> fields,
>>> >>> >>> >>>> >>> with keys that correspond to grid id and field name.  But
>>> >>> >>> >>>> >>> really,
>>> >>> >>> >>>> >>> it
>>> >>> >>> >>>> >>> could be any system, and I'm happy to work on
>>> >>> >>> >>>> >>> constructing
>>> >>> >>> >>>> >>> the
>>> >>> >>> >>>> >>> methods
>>> >>> >>> >>>> >>> to set this up.
>>> >>> >>> >>>> >>>
>>> >>> >>> >>>> >>> Best,
>>> >>> >>> >>>> >>>
>>> >>> >>> >>>> >>> Matt
>>> >>> >>> >>>> >>>
>>> >>> >>> >>>> >>> On Wed, May 25, 2011 at 5:54 PM, Jorge Poco
>>> >>> >>> >>>> >>> <jorge.poco at kitware.com>
>>> >>> >>> >>>> >>> wrote:
>>> >>> >>> >>>> >>>> Hi Matthew and Berk
>>> >>> >>> >>>> >>>>
>>> >>> >>> >>>> >>>> the full stack is
>>> >>> >>> >>>> >>>> here http://paste.enzotools.org/show/1668/ I'm
>>> >>> >>> >>>> >>>> trying to
>>> >>> >>> >>>> >>>> compile paraview with the VTK_USE_SYSTEM_HDF5 but It's
>>> >>> >>> >>>> >>>> giving me
>>> >>> >>> >>>> >>>> some
>>> >>> >>> >>>> >>>> errors, I will try again tomorrow.
>>> >>> >>> >>>> >>>>
>>> >>> >>> >>>> >>>> Matthew: Right now, I have written a 'programmable
>>> >>> >>> >>>> >>>> source'
>>> >>> >>> >>>> >>>> script to
>>> >>> >>> >>>> >>>> read a
>>> >>> >>> >>>> >>>> file from the system using the yt function 'load'. Then,
>>> >>> >>> >>>> >>>> create
>>> >>> >>> >>>> >>>> a
>>> >>> >>> >>>> >>>> projection
>>> >>> >>> >>>> >>>> using yt and then the ProjectionPlot object is converted
>>> >>> >>> >>>> >>>> into a
>>> >>> >>> >>>> >>>> vtkImageData
>>> >>> >>> >>>> >>>> to be displayed into paraview.
>>> >>> >>> >>>> >>>> What I want to do now, is to read a Dataset using
>>> >>> >>> >>>> >>>> paraview,
>>> >>> >>> >>>> >>>> then
>>> >>> >>> >>>> >>>> create a
>>> >>> >>> >>>> >>>> 'programmable filter' script to convert the data set
>>> >>> >>> >>>> >>>> into
>>> >>> >>> >>>> >>>>  yt's
>>> >>> >>> >>>> >>>> data
>>> >>> >>> >>>> >>>> structure to create a visualization and finally display
>>> >>> >>> >>>> >>>> the
>>> >>> >>> >>>> >>>> image
>>> >>> >>> >>>> >>>> inside
>>> >>> >>> >>>> >>>> paraview. What would be the correct way to create yt
>>> >>> >>> >>>> >>>> objects
>>> >>> >>> >>>> >>>> (EnzoHierarchy
>>> >>> >>> >>>> >>>> and EnzoGrid)?
>>> >>> >>> >>>> >>>>
>>> >>> >>> >>>> >>>> Berk: I was trying to find a way to get access to camera
>>> >>> >>> >>>> >>>> information
>>> >>> >>> >>>> >>>> from
>>> >>> >>> >>>> >>>> the 'programmable filter' to send this information to yt
>>> >>> >>> >>>> >>>>  but I
>>> >>> >>> >>>> >>>> didn't found
>>> >>> >>> >>>> >>>> anything useful. Is it possible?
>>> >>> >>> >>>> >>>>
>>> >>> >>> >>>> >>>>
>>> >>> >>> >>>> >>>> Regards,
>>> >>> >>> >>>> >>>>
>>> >>> >>> >>>> >>>> On Wed, May 25, 2011 at 3:10 PM, Berk Geveci
>>> >>> >>> >>>> >>>> <berk.geveci at kitware.com>
>>> >>> >>> >>>> >>>> wrote:
>>> >>> >>> >>>> >>>>>
>>> >>> >>> >>>> >>>>> If I understood what Matt is saying, the problem may be
>>> >>> >>> >>>> >>>>> fixed
>>> >>> >>> >>>> >>>>> by
>>> >>> >>> >>>> >>>>> having ParaView use an external HDF5, the one that yt
>>> >>> >>> >>>> >>>>> uses.
>>> >>> >>> >>>> >>>>> I
>>> >>> >>> >>>> >>>>> believe
>>> >>> >>> >>>> >>>>> that the variable is something like
>>> >>> >>> >>>> >>>>> PARAVIEW_USE_SYSTEM_HDF5
>>> >>> >>> >>>> >>>>> (or
>>> >>> >>> >>>> >>>>> VTK_,
>>> >>> >>> >>>> >>>>> not sure) in CMake.
>>> >>> >>> >>>> >>>>>
>>> >>> >>> >>>> >>>>> On Wed, May 25, 2011 at 10:49 AM, Matthew Turk
>>> >>> >>> >>>> >>>>> <matthewturk at gmail.com>
>>> >>> >>> >>>> >>>>> wrote:
>>> >>> >>> >>>> >>>>> > Hi Jorge,
>>> >>> >>> >>>> >>>>> >
>>> >>> >>> >>>> >>>>> > Wow, I didn't know that YT-embedding had taken off
>>> >>> >>> >>>> >>>>> > quite
>>> >>> >>> >>>> >>>>> > so
>>> >>> >>> >>>> >>>>> > much!
>>> >>> >>> >>>> >>>>> > This is great news.
>>> >>> >>> >>>> >>>>> >
>>> >>> >>> >>>> >>>>> > On Wed, May 25, 2011 at 9:48 AM, Jorge Poco
>>> >>> >>> >>>> >>>>> > <jorge.poco at kitware.com>
>>> >>> >>> >>>> >>>>> > wrote:
>>> >>> >>> >>>> >>>>> >> Hi
>>> >>> >>> >>>> >>>>> >> sorry my last email was sent incomplete
>>> >>> >>> >>>> >>>>> >> I have been trying to use YT inside the Paraview's
>>> >>> >>> >>>> >>>>> >> python
>>> >>> >>> >>>> >>>>> >> shell.
>>> >>> >>> >>>> >>>>> >> Initially I
>>> >>> >>> >>>> >>>>> >> have had some problems which are now identified. It
>>> >>> >>> >>>> >>>>> >> is
>>> >>> >>> >>>> >>>>> >> not
>>> >>> >>> >>>> >>>>> >> the
>>> >>> >>> >>>> >>>>> >> best
>>> >>> >>> >>>> >>>>> >> solution but I at least solve temporally these
>>> >>> >>> >>>> >>>>> >> problems:
>>> >>> >>> >>>> >>>>> >>
>>> >>> >>> >>>> >>>>> >> argv problem:
>>> >>> >>> >>>> >>>>> >>
>>> >>> >>> >>>> >>>>> >> Sentence: 'from yt.mods import *'
>>> >>> >>> >>>> >>>>> >> Error: for any reason paraview's python shell has
>>> >>> >>> >>>> >>>>> >> not
>>> >>> >>> >>>> >>>>> >> defined
>>> >>> >>> >>>> >>>>> >> the
>>> >>> >>> >>>> >>>>> >> variable
>>> >>> >>> >>>> >>>>> >> 'sys.argv'
>>> >>> >>> >>>> >>>>> >> Temporal solution: comment yt code where the
>>> >>> >>> >>>> >>>>> >> sys.argv
>>> >>> >>> >>>> >>>>> >> variable
>>> >>> >>> >>>> >>>>> >> is used:
>>> >>> >>> >>>> >>>>> >>
>>> >>> >>> >>>> >>>>> >> yt/funcs.py: lines 439-453
>>> >>> >>> >>>> >>>>> >>
>>> >>> >>> >>>> >>>>> >>
>>> >>> >>> >>>> >>>>> >> yt/utilities/parallel_tools/parallel_analysis_interface.py:
>>> >>> >>> >>>> >>>>> >> lines 47
>>> >>> >>> >>>> >>>>> >> and 48
>>> >>> >>> >>>> >>>>> >
>>> >>> >>> >>>> >>>>> > We can likely get around this on the yt side,
>>> >>> >>> >>>> >>>>> > although
>>> >>> >>> >>>> >>>>> > when
>>> >>> >>> >>>> >>>>> > embedding
>>> >>> >>> >>>> >>>>> > yt in Enzo we've escaped this by using the command
>>> >>> >>> >>>> >>>>> > PySys_SetArgv(argc,
>>> >>> >>> >>>> >>>>> > argv) immediately after calling Py_Initialize.  Does
>>> >>> >>> >>>> >>>>> > Partiview
>>> >>> >>> >>>> >>>>> > report
>>> >>> >>> >>>> >>>>> > a different executable name, by any chance?  If so we
>>> >>> >>> >>>> >>>>> > can
>>> >>> >>> >>>> >>>>> > use
>>> >>> >>> >>>> >>>>> > that as
>>> >>> >>> >>>> >>>>> > a signal to avoid requesting argv.
>>> >>> >>> >>>> >>>>> >
>>> >>> >>> >>>> >>>>> >>
>>> >>> >>> >>>> >>>>> >> :
>>> >>> >>> >>>> >>>>> >>
>>> >>> >>> >>>> >>>>> >> Sentence: "pf = load('tests/DD0010/moving7_0010')"
>>> >>> >>> >>>> >>>>> >> Error: python shell does not complete the load
>>> >>> >>> >>>> >>>>> >> function
>>> >>> >>> >>>> >>>>> >> and
>>> >>> >>> >>>> >>>>> >> paraview
>>> >>> >>> >>>> >>>>> >> keeps
>>> >>> >>> >>>> >>>>> >> waiting.
>>> >>> >>> >>>> >>>>> >> Temporal solution: When I debug paraview and
>>> >>> >>> >>>> >>>>> >> interrupt
>>> >>> >>> >>>> >>>>> >> its
>>> >>> >>> >>>> >>>>> >> execution
>>> >>> >>> >>>> >>>>> >> the
>>> >>> >>> >>>> >>>>> >> backtrace shows sem_wait(). It happens inside the
>>> >>> >>> >>>> >>>>> >> load
>>> >>> >>> >>>> >>>>> >> function
>>> >>> >>> >>>> >>>>> >> when
>>> >>> >>> >>>> >>>>> >> tries
>>> >>> >>> >>>> >>>>> >> to identify the candidates for input file
>>> >>> >>> >>>> >>>>> >> (convenience.py
>>> >>> >>> >>>> >>>>> >> line
>>> >>> >>> >>>> >>>>> >> 74).
>>> >>> >>> >>>> >>>>> >> There
>>> >>> >>> >>>> >>>>> >> are 11 possible candidates and gives error for 2 of
>>> >>> >>> >>>> >>>>> >> them:
>>> >>> >>> >>>> >>>>> >> ChomboStaticOutput
>>> >>> >>> >>>> >>>>> >> and FlashStaticOutput. Skipping those candidates
>>> >>> >>> >>>> >>>>> >> solve
>>> >>> >>> >>>> >>>>> >> temporally the
>>> >>> >>> >>>> >>>>> >> problem: (yt/convenienve.py line 75) inside the loop
>>> >>> >>> >>>> >>>>> >> add if
>>> >>> >>> >>>> >>>>> >> n ==
>>> >>> >>> >>>> >>>>> >> 'ChomboStaticOutput' or n == 'FlashStaticOutput':
>>> >>> >>> >>>> >>>>> >> continue;
>>> >>> >>> >>>> >>>>> >
>>> >>> >>> >>>> >>>>> > This is interesting.  What those two routines have in
>>> >>> >>> >>>> >>>>> > common
>>> >>> >>> >>>> >>>>> > is
>>> >>> >>> >>>> >>>>> > that
>>> >>> >>> >>>> >>>>> > they are the only two that try opening the file using
>>> >>> >>> >>>> >>>>> > h5py.
>>> >>> >>> >>>> >>>>> >  HDF5
>>> >>> >>> >>>> >>>>> > has
>>> >>> >>> >>>> >>>>> > a number of error handling routines, which in the
>>> >>> >>> >>>> >>>>> > past
>>> >>> >>> >>>> >>>>> > I've
>>> >>> >>> >>>> >>>>> > seen
>>> >>> >>> >>>> >>>>> > issues with when embedding one app in another -- h5py
>>> >>> >>> >>>> >>>>> > for
>>> >>> >>> >>>> >>>>> > instance
>>> >>> >>> >>>> >>>>> > handles error handling, but that can conflict with
>>> >>> >>> >>>> >>>>> > other
>>> >>> >>> >>>> >>>>> > mechanisms
>>> >>> >>> >>>> >>>>> > that are attempting to handle those errors.
>>> >>> >>> >>>> >>>>> >
>>> >>> >>> >>>> >>>>> > Another solution that would avoid digging into the
>>> >>> >>> >>>> >>>>> > source
>>> >>> >>> >>>> >>>>> > code
>>> >>> >>> >>>> >>>>> > would
>>> >>> >>> >>>> >>>>> > be to remove the types from the actual type registry:
>>> >>> >>> >>>> >>>>> >
>>> >>> >>> >>>> >>>>> > from yt.convenience import output_type_registry
>>> >>> >>> >>>> >>>>> > del output_type_registry["ChomboStaticOutput"]
>>> >>> >>> >>>> >>>>> > del output_type_registry["FlashStaticOutput"]
>>> >>> >>> >>>> >>>>> >
>>> >>> >>> >>>> >>>>> > I'm more interested in figuring out how to fix the
>>> >>> >>> >>>> >>>>> > interplay
>>> >>> >>> >>>> >>>>> > between
>>> >>> >>> >>>> >>>>> > hdf5-paraview and h5py, though.  Do you happen to
>>> >>> >>> >>>> >>>>> > have a
>>> >>> >>> >>>> >>>>> > full
>>> >>> >>> >>>> >>>>> > stack
>>> >>> >>> >>>> >>>>> > trace?
>>> >>> >>> >>>> >>>>> >
>>> >>> >>> >>>> >>>>> > Thanks very much for these reports -- this is very
>>> >>> >>> >>>> >>>>> > exciting,
>>> >>> >>> >>>> >>>>> > and
>>> >>> >>> >>>> >>>>> > I'm
>>> >>> >>> >>>> >>>>> > eager to help out!
>>> >>> >>> >>>> >>>>> >
>>> >>> >>> >>>> >>>>> > Best,
>>> >>> >>> >>>> >>>>> >
>>> >>> >>> >>>> >>>>> > Matt
>>> >>> >>> >>>> >>>>> >
>>> >>> >>> >>>> >>>>> >>
>>> >>> >>> >>>> >>>>> >> Any of you have a better way to solve the problems
>>> >>> >>> >>>> >>>>> >> above?
>>> >>> >>> >>>> >>>>> >> Regards,
>>> >>> >>> >>>> >>>>> >> On Wed, May 25, 2011 at 9:43 AM, Jorge Poco
>>> >>> >>> >>>> >>>>> >> <jorge.poco at kitware.com>
>>> >>> >>> >>>> >>>>> >> wrote:
>>> >>> >>> >>>> >>>>> >>>
>>> >>> >>> >>>> >>>>> >>> Hi
>>> >>> >>> >>>> >>>>> >>> I have been trying to use YT inside the Paraview's
>>> >>> >>> >>>> >>>>> >>> python
>>> >>> >>> >>>> >>>>> >>> shell.
>>> >>> >>> >>>> >>>>> >>> Initially
>>> >>> >>> >>>> >>>>> >>> I have had some problems which are now identified.
>>> >>> >>> >>>> >>>>> >>> It
>>> >>> >>> >>>> >>>>> >>> is
>>> >>> >>> >>>> >>>>> >>> not
>>> >>> >>> >>>> >>>>> >>> the
>>> >>> >>> >>>> >>>>> >>> smart solution but I at least solve temporally
>>> >>> >>> >>>> >>>>> >>> these
>>> >>> >>> >>>> >>>>> >>> problems:
>>> >>> >>> >>>> >>>>> >>>
>>> >>> >>> >>>> >>>>> >>> argv problem:
>>> >>> >>> >>>> >>>>> >>>
>>> >>> >>> >>>> >>>>> >>> Sentence: 'from yt.mods import *'
>>> >>> >>> >>>> >>>>> >>> Error: for any reason paraview's python shell has
>>> >>> >>> >>>> >>>>> >>> not
>>> >>> >>> >>>> >>>>> >>> defined
>>> >>> >>> >>>> >>>>> >>> the
>>> >>> >>> >>>> >>>>> >>> variable
>>> >>> >>> >>>> >>>>> >>> 'sys.argv'
>>> >>> >>> >>>> >>>>> >>> Temporal solution: comment yt code where the
>>> >>> >>> >>>> >>>>> >>> sys.argv
>>> >>> >>> >>>> >>>>> >>> variable
>>> >>> >>> >>>> >>>>> >>> is
>>> >>> >>> >>>> >>>>> >>> used:
>>> >>> >>> >>>> >>>>> >>>
>>> >>> >>> >>>> >>>>> >>> yt/funcs.py: lines 439-453
>>> >>> >>> >>>> >>>>> >>>
>>> >>> >>> >>>> >>>>> >>>
>>> >>> >>> >>>> >>>>> >>> yt/utilities/parallel_tools/parallel_analysis_interface.py:
>>> >>> >>> >>>> >>>>> >>> lines 47
>>> >>> >>> >>>> >>>>> >>> and
>>> >>> >>> >>>> >>>>> >>> 48
>>> >>> >>> >>>> >>>>> >>>
>>> >>> >>> >>>> >>>>> >>> pa:
>>> >>> >>> >>>> >>>>> >>>
>>> >>> >>> >>>> >>>>> >>> Sentence: "pf = load('tests/DD0010/moving7_0010')"
>>> >>> >>> >>>> >>>>> >>> Error: python shell does not complete the load
>>> >>> >>> >>>> >>>>> >>> function
>>> >>> >>> >>>> >>>>> >>> and
>>> >>> >>> >>>> >>>>> >>> paraview
>>> >>> >>> >>>> >>>>> >>> keeps
>>> >>> >>> >>>> >>>>> >>> waiting.
>>> >>> >>> >>>> >>>>> >>> Temporal solution: When I debug paraview and
>>> >>> >>> >>>> >>>>> >>> interrupt
>>> >>> >>> >>>> >>>>> >>> its
>>> >>> >>> >>>> >>>>> >>> execution
>>> >>> >>> >>>> >>>>> >>> the
>>> >>> >>> >>>> >>>>> >>> backtrace shows sem_wait(). It happens inside the
>>> >>> >>> >>>> >>>>> >>> load
>>> >>> >>> >>>> >>>>> >>> function
>>> >>> >>> >>>> >>>>> >>> when
>>> >>> >>> >>>> >>>>> >>> tries
>>> >>> >>> >>>> >>>>> >>> to identify the candidates for input file
>>> >>> >>> >>>> >>>>> >>> (convenience.py
>>> >>> >>> >>>> >>>>> >>> line
>>> >>> >>> >>>> >>>>> >>> 74).
>>> >>> >>> >>>> >>>>> >>> There
>>> >>> >>> >>>> >>>>> >>> are 11 possible candidates and gives error for 2 of
>>> >>> >>> >>>> >>>>> >>> them:
>>> >>> >>> >>>> >>>>> >>> ChomboStaticOutput
>>> >>> >>> >>>> >>>>> >>> and FlashStaticOutput. Skipping those candidates
>>> >>> >>> >>>> >>>>> >>> solve
>>> >>> >>> >>>> >>>>> >>> temporally the
>>> >>> >>> >>>> >>>>> >>> problem:
>>> >>> >>> >>>> >>>>> >>
>>> >>> >>> >>>> >>>>> >>
>>> >>> >>> >>>> >>>>> >> _______________________________________________
>>> >>> >>> >>>> >>>>> >> Amr mailing list
>>> >>> >>> >>>> >>>>> >> Amr at public.kitware.com
>>> >>> >>> >>>> >>>>> >>
>>> >>> >>> >>>> >>>>> >> http://public.kitware.com/cgi-bin/mailman/listinfo/amr
>>> >>> >>> >>>> >>>>> >>
>>> >>> >>> >>>> >>>>> >>
>>> >>> >>> >>>> >>>>> > _______________________________________________
>>> >>> >>> >>>> >>>>> > Amr mailing list
>>> >>> >>> >>>> >>>>> > Amr at public.kitware.com
>>> >>> >>> >>>> >>>>> >
>>> >>> >>> >>>> >>>>> > http://public.kitware.com/cgi-bin/mailman/listinfo/amr
>>> >>> >>> >>>> >>>>> >
>>> >>> >>> >>>> >>>>> _______________________________________________
>>> >>> >>> >>>> >>>>> Amr mailing list
>>> >>> >>> >>>> >>>>> Amr at public.kitware.com
>>> >>> >>> >>>> >>>>> http://public.kitware.com/cgi-bin/mailman/listinfo/amr
>>> >>> >>> >>>> >>>>
>>> >>> >>> >>>> >>>>
>>> >>> >>> >>>> >>>> _______________________________________________
>>> >>> >>> >>>> >>>> Amr mailing list
>>> >>> >>> >>>> >>>> Amr at public.kitware.com
>>> >>> >>> >>>> >>>> http://public.kitware.com/cgi-bin/mailman/listinfo/amr
>>> >>> >>> >>>> >>>>
>>> >>> >>> >>>> >>>>
>>> >>> >>> >>>> >>> _______________________________________________
>>> >>> >>> >>>> >>> Amr mailing list
>>> >>> >>> >>>> >>> Amr at public.kitware.com
>>> >>> >>> >>>> >>> http://public.kitware.com/cgi-bin/mailman/listinfo/amr
>>> >>> >>> >>>> >>>
>>> >>> >>> >>>> >>
>>> >>> >>> >>>> > _______________________________________________
>>> >>> >>> >>>> > Amr mailing list
>>> >>> >>> >>>> > Amr at public.kitware.com
>>> >>> >>> >>>> > http://public.kitware.com/cgi-bin/mailman/listinfo/amr
>>> >>> >>> >>>> >
>>> >>> >>> >>>> _______________________________________________
>>> >>> >>> >>>> Amr mailing list
>>> >>> >>> >>>> Amr at public.kitware.com
>>> >>> >>> >>>> http://public.kitware.com/cgi-bin/mailman/listinfo/amr
>>> >>> >>> >>>
>>> >>> >>> >>
>>> >>> >>> >>
>>> >>> >>> >
>>> >>> >>> _______________________________________________
>>> >>> >>> Amr mailing list
>>> >>> >>> Amr at public.kitware.com
>>> >>> >>> http://public.kitware.com/cgi-bin/mailman/listinfo/amr
>>> >>> >>
>>> >>> >>
>>> >>> > _______________________________________________
>>> >>> > Amr mailing list
>>> >>> > Amr at public.kitware.com
>>> >>> > http://public.kitware.com/cgi-bin/mailman/listinfo/amr
>>> >>> >
>>> >>> _______________________________________________
>>> >>> Amr mailing list
>>> >>> Amr at public.kitware.com
>>> >>> http://public.kitware.com/cgi-bin/mailman/listinfo/amr
>>> >>
>>> >>
>>> >
>>
>>
>