[Amr] yt inside paraview

Mon Jun 13 11:23:43 EDT 2011

Hi Matt

I have the previous ProgrammablerSource with this ProgrammableFilter, and
now, we can should be able to use the dataset loaded with Paraview apply
yt's visualization algorihtms and visualize in Paraview, which is really
cool :D

The code is here http://paste.enzotools.org/show/1703/

I have changed: yt/data_container.py:1539
from return QuadTree(na.array([xd,yd]), nvals)
to     return QuadTree(na.array([xd,yd], dtype='int64'), nvals)
Because it was giving me some error in the add_projection function.

Regards,

On Fri, Jun 10, 2011 at 5:34 PM, Matthew Turk <matthewturk at gmail.com> wrote:

> Hi Jorge,
>
> Here's what I came up with.  I tested it by opening an Enzo dataset,
> loading a few fields, and then creating a programmable filter.  It
> looks like it received everything.
>
> http://paste.enzotools.org/show/1698/
>
> This is exciting!  It actually gave me good output.  So I suppose the
> next question is, how should we proceed?  Do you think it's time to
> start cleaning up the yt interface, as it does seem a bit cumbersome?
> Or should we try to, for instance, create new derived fields yt knows
> how to make?
>
> Thanks,
>
> Matt
>
> On Fri, Jun 10, 2011 at 1:48 PM, Matthew Turk <matthewturk at gmail.com>
> wrote:
> > Hi Jorge,
> >
> > I'm still working on the parent/child relationship.  I've implemented
> > a stopgap that will accept "None" an then reconstruct the values; I
> > recognize this is inefficient and I will replace it.
> >
> > The other thing is that now if you supply an argument "io" to the
> > StreamHandler, it will use that instead of the previous method for
> > stream io handling.
> >
> > This object should subclass yt.utilities.io_handler.BaseIOHandler and
> > should implement the following methods:
> >
> > _read_data_set(self, grid, field)
> > _read_field_names(self, grid)
> > _read_data_slice(self, grid, field, axis, coord) # coord is an int
> >
> > It should also define a _data_style string attribute; in the future,
> > once we've settled this down, we'll wrap the paraview IO style
> > directly in and make this a natively understood method.  It should
> > also define a _read_exception, which is used to tell if there's been a
> > problem and yt should try to generate a field instead of reading it
> > from disk.
> >
> > I'm going try to wrap what you've written into this framework and load
> > it up here.  I'll ping the list and update the examples on the wiki
> > page when that's done.
> >
> > -Matt
> >
> > These are all in 48ee9a3a6e4a in the bitbucket repo.
> >
> > On Fri, Jun 10, 2011 at 12:59 PM, Jorge Poco <jorge.poco at kitware.com>
> wrote:
> >> Hi Matt
> >> I have written the StreamFieldHandler and StreamHandler to interact with
> >> Paraview, but I have a couple of questions:
> >>
> >> parent_ids will be change by child masking, right?
> >> In the function __getitem__ of StreamFieldHandler, could be changed to
> >> receive 2 parameter instead of 1?, these parameters would be index_grid
> and
> >> field_name,  and should return an numpy.array in 3D. In way it's done
> now
> >> could work, but the array returned will be 1D, so, I think I could
> handle it
> >> in __getitem__ and return the correct array.
> >>
> >> The ProgrammableFilter filter is
> >> here http://paste.enzotools.org/show/1695/ I was using
> >> the tests/DD0010/moving7_0010.hierarchy data set.
> >> Regards,
> >>
> >> On Fri, Jun 10, 2011 at 3:21 PM, Matthew Turk <matthewturk at gmail.com>
> wrote:
> >>>
> >>> As a quick note, the values should all differ by single multiplicative
> >>> factor for each field.
> >>>
> >>> On Fri, Jun 10, 2011 at 12:19 PM, Matthew Turk <matthewturk at gmail.com>
> >>> wrote:
> >>> > Hi Jorge,
> >>> >
> >>> > Yup, we apply the unit conversions to get them back to CGS.
> >>> > Temperature field takes no conversions.
> >>> >
> >>> > -Matt
> >>> >
> >>> > On Fri, Jun 10, 2011 at 12:18 PM, Jorge Poco <jorge.poco at kitware.com
> >
> >>> > wrote:
> >>> >> HI Matt
> >>> >> Another doubt: I'm using the tests/DD0010/moving7_0010 data set. I
> have
> >>> >> tried to load it using YT and Paraview.
> >>> >> The problem is that grid values are different in both tools except
> >>> >> temperature field.  I'm wondering if you
> >>> >> do any conversions in the grid values when you load the data set.
> >>> >> The code I'm using in yt is:
> >>> >>>> pf =  load('tests/DD0010/moving7_0010')
> >>> >>>> pf.h.grids[0]['Density'].min()
> >>> >>>> pf.h.grids[0]['Density'].max()
> >>> >> Regards,
> >>> >> On Fri, Jun 10, 2011 at 2:40 PM, George Zagaris
> >>> >> <george.zagaris at kitware.com>
> >>> >> wrote:
> >>> >>>
> >>> >>> That's great!
> >>> >>>
> >>> >>> Yes the vtkHierarchicalBoxDataSet has IBLANK information that
> >>> >>> can be supplied directly to yt.
> >>> >>>
> >>> >>> Best,
> >>> >>> George
> >>> >>>
> >>> >>> On Fri, Jun 10, 2011 at 2:37 PM, Matthew Turk <
> matthewturk at gmail.com>
> >>> >>> wrote:
> >>> >>> > Hi Jorge,
> >>> >>> >
> >>> >>> > Nope, it should not be mandatory, although it currently is.  It
> is
> >>> >>> > currently only used to calculate child masking, which I believe
> we
> >>> >>> > can
> >>> >>> > also get from vtkHierarchicalBoxDataSet?  I'll make this change
> >>> >>> > later
> >>> >>> > today.
> >>> >>> >
> >>> >>> > -Matt
> >>> >>> >
> >>> >>> >
> >>> >>> >
> >>> >>> > On Fri, Jun 10, 2011 at 11:35 AM, Jorge Poco
> >>> >>> > <jorge.poco at kitware.com>
> >>> >>> > wrote:
> >>> >>> >> HI Matt
> >>> >>> >> Looks nice.
> >>> >>> >> In the 'stream' frontends, when the stream.StreamHandler(...) is
> >>> >>> >> create,
> >>> >>> >> one parameter is the parent_ids right? It's really mandatory to
> >>> >>> >> send
> >>> >>> >> it?
> >>> >>> >> because the vtkHierarchicalBoxDataSet doesn't save this
> >>> >>> >> information,
> >>> >>> >> but we
> >>> >>> >> can compute the parent/child relations.
> >>> >>> >> Regards,
> >>> >>> >>
> >>> >>> >> On Fri, Jun 10, 2011 at 2:19 PM, Matthew Turk
> >>> >>> >> <matthewturk at gmail.com>
> >>> >>> >> wrote:
> >>> >>> >>>
> >>> >>> >>> Hi Jorge,
> >>> >>> >>>
> >>> >>> >>> I thought this might be of interest to you; I've started
> keeping
> >>> >>> >>> track
> >>> >>> >>> of how to interface with external codes such as ParaView here:
> >>> >>> >>>
> >>> >>> >>> http://hg.enzotools.org/yt/wiki/InterfacingWithYT
> >>> >>> >>>
> >>> >>> >>> I'll be updating this as need be with information about the
> >>> >>> >>> in-memory
> >>> >>> >>> datasets and so on.
> >>> >>> >>>
> >>> >>> >>> Best,
> >>> >>> >>>
> >>> >>> >>> Matt
> >>> >>> >>>
> >>> >>> >>> On Fri, May 27, 2011 at 2:08 PM, Matthew Turk
> >>> >>> >>> <matthewturk at gmail.com>
> >>> >>> >>> wrote:
> >>> >>> >>> > Hi all,
> >>> >>> >>> >
> >>> >>> >>> > I've created a new data frontend in yt entitled "Stream" (the
> >>> >>> >>> > name
> >>> >>> >>> > is
> >>> >>> >>> > up for discussion, if someone has a better idea -- Charles
> >>> >>> >>> > pointed
> >>> >>> >>> > out
> >>> >>> >>> > today that it might get confused with time series data.)  The
> >>> >>> >>> > idea
> >>> >>> >>> > behind this is that the user can use it to construct an
> >>> >>> >>> > arbitrary
> >>> >>> >>> > dataset in memory.  I've tested this by proxying an existing
> >>> >>> >>> > Enzo
> >>> >>> >>> > dataset:
> >>> >>> >>> >
> >>> >>> >>> >
> >>> >>> >>> >
> >>> >>> >>> >
> >>> >>> >>> >
> http://matthewturk.bitbucket.org/html/ba3fd37b-842c-4641-b21e-2d3f5268eefe-stream_proxy-py.html
> >>> >>> >>> >
> >>> >>> >>> > (downloadable at the top level domain, or by "yt_lodgeit.py
> >>> >>> >>> > --download=1672 if you have a yt stack installed.)
> >>> >>> >>> >
> >>> >>> >>> > You have to construct both a StreamHandler that knows how to
> >>> >>> >>> > pass
> >>> >>> >>> > around the grid geometry and which, although not instantiated
> >>> >>> >>> > with
> >>> >>> >>> > them, requires certain attributes to be set.  In addition, a
> >>> >>> >>> > fields
> >>> >>> >>> > handler has to be created.  This script executes to
> completion
> >>> >>> >>> > and
> >>> >>> >>> > passes its tests; projecting and slicing require slightly
> more
> >>> >>> >>> > effort,
> >>> >>> >>> > because I wanted to pose to the Kitware group how the data
> >>> >>> >>> > should be
> >>> >>> >>> > evaluated in terms of units -- should it be assumed to have
> CGS
> >>> >>> >>> > units
> >>> >>> >>> > already?  How should the domain size be handled -- in yt
> right
> >>> >>> >>> > now
> >>> >>> >>> > units are distinct from the code definitions, but I don't
> know
> >>> >>> >>> > how
> >>> >>> >>> > Paraview handles this distinction.  (If you want to try
> setting
> >>> >>> >>> > up a
> >>> >>> >>> > PlotCollection, you can, but you will have to set
> >>> >>> >>> > spf.units["cm"] =
> >>> >>> >>> > spf.units["unitary"] = 1.0).
> >>> >>> >>> >
> >>> >>> >>> > Hopefully this can be a start, and we can continue to iterate
> on
> >>> >>> >>> > the
> >>> >>> >>> > API necessary to load data in yt from that already loaded in
> >>> >>> >>> > Paraview.
> >>> >>> >>> >  This is in the tip of the development yt repository,
> >>> >>> >>> > 6d4a5588a550.
> >>> >>> >>> >
> >>> >>> >>> > Best,
> >>> >>> >>> >
> >>> >>> >>> > Matt
> >>> >>> >>> >
> >>> >>> >>> > On Thu, May 26, 2011 at 3:54 PM, Jorge Poco
> >>> >>> >>> > <jorge.poco at kitware.com>
> >>> >>> >>> > wrote:
> >>> >>> >>> >> In the  meantime I will be working to import yt's data
> >>> >>> >>> >> structures
> >>> >>> >>> >> inside
> >>> >>> >>> >> vtkHierarchicalBoxDataset using ProgrammableSources inside
> >>> >>> >>> >> Python.
> >>> >>> >>> >> Regards,
> >>> >>> >>> >>
> >>> >>> >>> >> On Thu, May 26, 2011 at 3:43 PM, Jorge Poco
> >>> >>> >>> >> <jorge.poco at kitware.com>
> >>> >>> >>> >> wrote:
> >>> >>> >>> >>>
> >>> >>> >>> >>> Hi
> >>> >>> >>> >>> YT has some function to access the arrays data in
> numpy.array
> >>> >>> >>> >>> objects
> >>> >>> >>> >>> (
> >>> >>> >>> >>> pf.h.grids[0]['Density']), using them  we can instantiate a
> >>> >>> >>> >>> vtkHierarchicalDataset to be process into paraview.
> >>> >>> >>> >>> On Thu, May 26, 2011 at 2:53 PM, Berk Geveci
> >>> >>> >>> >>> <berk.geveci at kitware.com>
> >>> >>> >>> >>> wrote:
> >>> >>> >>> >>>>
> >>> >>> >>> >>>> Awesome.
> >>> >>> >>> >>>>
> >>> >>> >>> >>>> What about from the other side? If we wanted to pass data
> >>> >>> >>> >>>> structures
> >>> >>> >>> >>>> that yt loaded (from an Enzo file for example) to
> >>> >>> >>> >>>> VTK/ParaView
> >>> >>> >>> >>>> for
> >>> >>> >>> >>>> filtering/rendering?
> >>> >>> >>> >>>>
> >>> >>> >>> >>>> -berk
> >>> >>> >>> >>>>
> >>> >>> >>> >>>> On Thu, May 26, 2011 at 2:04 PM, Matthew Turk
> >>> >>> >>> >>>> <matthewturk at gmail.com>
> >>> >>> >>> >>>> wrote:
> >>> >>> >>> >>>> > Hi Berk and Jorge,
> >>> >>> >>> >>>> >
> >>> >>> >>> >>>> > Okay, this makes sense.
> >>> >>> >>> >>>> >
> >>> >>> >>> >>>> > I will write up a new code frontend for yt that will act
> as
> >>> >>> >>> >>>> > a
> >>> >>> >>> >>>> > generalized system; this will draw upon the current in
> situ
> >>> >>> >>> >>>> > Enzo
> >>> >>> >>> >>>> > frontend, and will be designed to be instantiated with a
> >>> >>> >>> >>>> > list
> >>> >>> >>> >>>> > of
> >>> >>> >>> >>>> > arrays and a set of callback functions.  We can them
> >>> >>> >>> >>>> > implement
> >>> >>> >>> >>>> > the
> >>> >>> >>> >>>> > paraview-specific code inside this, or perhaps inside a
> >>> >>> >>> >>>> > ProgrammableFilter inside paraview itself.  I'll ping
> the
> >>> >>> >>> >>>> > list
> >>> >>> >>> >>>> > back
> >>> >>> >>> >>>> > when I have something up and running.
> >>> >>> >>> >>>> >
> >>> >>> >>> >>>> > Best,
> >>> >>> >>> >>>> >
> >>> >>> >>> >>>> > Matt
> >>> >>> >>> >>>> >
> >>> >>> >>> >>>> > On Thu, May 26, 2011 at 9:15 AM, Berk Geveci
> >>> >>> >>> >>>> > <berk.geveci at kitware.com>
> >>> >>> >>> >>>> > wrote:
> >>> >>> >>> >>>> >> In VTK, an AMR mesh is represented as a collection of
> >>> >>> >>> >>>> >> vtkUniformGrid
> >>> >>> >>> >>>> >> objects. Each vtkUniformGrid has a set of
> Fortran-ordered
> >>> >>> >>> >>>> >> data
> >>> >>> >>> >>>> >> arrays
> >>> >>> >>> >>>> >> that represent the variables. One exception is how
> vectors
> >>> >>> >>> >>>> >> are
> >>> >>> >>> >>>> >> stored.
> >>> >>> >>> >>>> >> They are stored in hybrid Fortran - C ordering, i.e.
> >>> >>> >>> >>>> >> vector
> >>> >>> >>> >>>> >> component
> >>> >>> >>> >>>> >> increases fastest then i, then j, then k. It is
> >>> >>> >>> >>>> >> straightforward to
> >>> >>> >>> >>>> >> pass these to Python and back using NumPy arrays or
> buffer
> >>> >>> >>> >>>> >> objects
> >>> >>> >>> >>>> >> at
> >>> >>> >>> >>>> >> the lowest level. So we can write Python code to bundle
> >>> >>> >>> >>>> >> these
> >>> >>> >>> >>>> >> arrays
> >>> >>> >>> >>>> >> into any form that yt (and Enzo when we start doing
> >>> >>> >>> >>>> >> in-situ)
> >>> >>> >>> >>>> >> wants. We
> >>> >>> >>> >>>> >> may have to deep-copy the vectors because of the issue
> I
> >>> >>> >>> >>>> >> described.
> >>> >>> >>> >>>> >>
> >>> >>> >>> >>>> >> On Wed, May 25, 2011 at 7:59 PM, Matthew Turk
> >>> >>> >>> >>>> >> <matthewturk at gmail.com>
> >>> >>> >>> >>>> >> wrote:
> >>> >>> >>> >>>> >>> Hi Jorge,
> >>> >>> >>> >>>> >>>
> >>> >>> >>> >>>> >>> This is really cool!
> >>> >>> >>> >>>> >>>
> >>> >>> >>> >>>> >>> I've been thinking about how to invert this process,
> and
> >>> >>> >>> >>>> >>> I
> >>> >>> >>> >>>> >>> think
> >>> >>> >>> >>>> >>> we
> >>> >>> >>> >>>> >>> could address it by creating a new hierarchy of
> objects,
> >>> >>> >>> >>>> >>> similar
> >>> >>> >>> >>>> >>> to
> >>> >>> >>> >>>> >>> the EnzoStaticOutputInMemory family of objects, that
> >>> >>> >>> >>>> >>> enables
> >>> >>> >>> >>>> >>> a
> >>> >>> >>> >>>> >>> bi-directional transfer of information.  All that's
> >>> >>> >>> >>>> >>> really
> >>> >>> >>> >>>> >>> necessary
> >>> >>> >>> >>>> >>> to get data in is to describe the mesh, describe the
> >>> >>> >>> >>>> >>> parameters
> >>> >>> >>> >>>> >>> of
> >>> >>> >>> >>>> >>> the
> >>> >>> >>> >>>> >>> simulation, and then communicate to yt how to access
> >>> >>> >>> >>>> >>> data.
> >>> >>> >>> >>>> >>>
> >>> >>> >>> >>>> >>> What do you think the best mechanism for passing data
> >>> >>> >>> >>>> >>> back
> >>> >>> >>> >>>> >>> and
> >>> >>> >>> >>>> >>> forth
> >>> >>> >>> >>>> >>> is?  For in situ viz with Enzo, usually we do this by
> >>> >>> >>> >>>> >>> constructing a
> >>> >>> >>> >>>> >>> new module object in the C code, and then insert into
> >>> >>> >>> >>>> >>> that a
> >>> >>> >>> >>>> >>> set
> >>> >>> >>> >>>> >>> of
> >>> >>> >>> >>>> >>> arrays describing the mesh and a dictionary of
> >>> >>> >>> >>>> >>> dictionaries
> >>> >>> >>> >>>> >>> of
> >>> >>> >>> >>>> >>> fields,
> >>> >>> >>> >>>> >>> with keys that correspond to grid id and field name.
>  But
> >>> >>> >>> >>>> >>> really,
> >>> >>> >>> >>>> >>> it
> >>> >>> >>> >>>> >>> could be any system, and I'm happy to work on
> >>> >>> >>> >>>> >>> constructing
> >>> >>> >>> >>>> >>> the
> >>> >>> >>> >>>> >>> methods
> >>> >>> >>> >>>> >>> to set this up.
> >>> >>> >>> >>>> >>>
> >>> >>> >>> >>>> >>> Best,
> >>> >>> >>> >>>> >>>
> >>> >>> >>> >>>> >>> Matt
> >>> >>> >>> >>>> >>>
> >>> >>> >>> >>>> >>> On Wed, May 25, 2011 at 5:54 PM, Jorge Poco
> >>> >>> >>> >>>> >>> <jorge.poco at kitware.com>
> >>> >>> >>> >>>> >>> wrote:
> >>> >>> >>> >>>> >>>> Hi Matthew and Berk
> >>> >>> >>> >>>> >>>>
> >>> >>> >>> >>>> >>>> the full stack is
> >>> >>> >>> >>>> >>>> here http://paste.enzotools.org/show/1668/ I'm
> >>> >>> >>> >>>> >>>> trying to
> >>> >>> >>> >>>> >>>> compile paraview with the VTK_USE_SYSTEM_HDF5 but
> It's
> >>> >>> >>> >>>> >>>> giving me
> >>> >>> >>> >>>> >>>> some
> >>> >>> >>> >>>> >>>> errors, I will try again tomorrow.
> >>> >>> >>> >>>> >>>>
> >>> >>> >>> >>>> >>>> Matthew: Right now, I have written a 'programmable
> >>> >>> >>> >>>> >>>> source'
> >>> >>> >>> >>>> >>>> script to
> >>> >>> >>> >>>> >>>> read a
> >>> >>> >>> >>>> >>>> file from the system using the yt function 'load'.
> Then,
> >>> >>> >>> >>>> >>>> create
> >>> >>> >>> >>>> >>>> a
> >>> >>> >>> >>>> >>>> projection
> >>> >>> >>> >>>> >>>> using yt and then the ProjectionPlot object is
> converted
> >>> >>> >>> >>>> >>>> into a
> >>> >>> >>> >>>> >>>> vtkImageData
> >>> >>> >>> >>>> >>>> to be displayed into paraview.
> >>> >>> >>> >>>> >>>> What I want to do now, is to read a Dataset using
> >>> >>> >>> >>>> >>>> paraview,
> >>> >>> >>> >>>> >>>> then
> >>> >>> >>> >>>> >>>> create a
> >>> >>> >>> >>>> >>>> 'programmable filter' script to convert the data set
> >>> >>> >>> >>>> >>>> into
> >>> >>> >>> >>>> >>>>  yt's
> >>> >>> >>> >>>> >>>> data
> >>> >>> >>> >>>> >>>> structure to create a visualization and finally
> display
> >>> >>> >>> >>>> >>>> the
> >>> >>> >>> >>>> >>>> image
> >>> >>> >>> >>>> >>>> inside
> >>> >>> >>> >>>> >>>> paraview. What would be the correct way to create yt
> >>> >>> >>> >>>> >>>> objects
> >>> >>> >>> >>>> >>>> (EnzoHierarchy
> >>> >>> >>> >>>> >>>> and EnzoGrid)?
> >>> >>> >>> >>>> >>>>
> >>> >>> >>> >>>> >>>> Berk: I was trying to find a way to get access to
> camera
> >>> >>> >>> >>>> >>>> information
> >>> >>> >>> >>>> >>>> from
> >>> >>> >>> >>>> >>>> the 'programmable filter' to send this information to
> yt
> >>> >>> >>> >>>> >>>>  but I
> >>> >>> >>> >>>> >>>> didn't found
> >>> >>> >>> >>>> >>>> anything useful. Is it possible?
> >>> >>> >>> >>>> >>>>
> >>> >>> >>> >>>> >>>>
> >>> >>> >>> >>>> >>>> Regards,
> >>> >>> >>> >>>> >>>>
> >>> >>> >>> >>>> >>>> On Wed, May 25, 2011 at 3:10 PM, Berk Geveci
> >>> >>> >>> >>>> >>>> <berk.geveci at kitware.com>
> >>> >>> >>> >>>> >>>> wrote:
> >>> >>> >>> >>>> >>>>>
> >>> >>> >>> >>>> >>>>> If I understood what Matt is saying, the problem may
> be
> >>> >>> >>> >>>> >>>>> fixed
> >>> >>> >>> >>>> >>>>> by
> >>> >>> >>> >>>> >>>>> having ParaView use an external HDF5, the one that
> yt
> >>> >>> >>> >>>> >>>>> uses.
> >>> >>> >>> >>>> >>>>> I
> >>> >>> >>> >>>> >>>>> believe
> >>> >>> >>> >>>> >>>>> that the variable is something like
> >>> >>> >>> >>>> >>>>> PARAVIEW_USE_SYSTEM_HDF5
> >>> >>> >>> >>>> >>>>> (or
> >>> >>> >>> >>>> >>>>> VTK_,
> >>> >>> >>> >>>> >>>>> not sure) in CMake.
> >>> >>> >>> >>>> >>>>>
> >>> >>> >>> >>>> >>>>> On Wed, May 25, 2011 at 10:49 AM, Matthew Turk
> >>> >>> >>> >>>> >>>>> <matthewturk at gmail.com>
> >>> >>> >>> >>>> >>>>> wrote:
> >>> >>> >>> >>>> >>>>> > Hi Jorge,
> >>> >>> >>> >>>> >>>>> >
> >>> >>> >>> >>>> >>>>> > Wow, I didn't know that YT-embedding had taken off
> >>> >>> >>> >>>> >>>>> > quite
> >>> >>> >>> >>>> >>>>> > so
> >>> >>> >>> >>>> >>>>> > much!
> >>> >>> >>> >>>> >>>>> > This is great news.
> >>> >>> >>> >>>> >>>>> >
> >>> >>> >>> >>>> >>>>> > On Wed, May 25, 2011 at 9:48 AM, Jorge Poco
> >>> >>> >>> >>>> >>>>> > <jorge.poco at kitware.com>
> >>> >>> >>> >>>> >>>>> > wrote:
> >>> >>> >>> >>>> >>>>> >> Hi
> >>> >>> >>> >>>> >>>>> >> sorry my last email was sent incomplete
> >>> >>> >>> >>>> >>>>> >> I have been trying to use YT inside the
> Paraview's
> >>> >>> >>> >>>> >>>>> >> python
> >>> >>> >>> >>>> >>>>> >> shell.
> >>> >>> >>> >>>> >>>>> >> Initially I
> >>> >>> >>> >>>> >>>>> >> have had some problems which are now identified.
> It
> >>> >>> >>> >>>> >>>>> >> is
> >>> >>> >>> >>>> >>>>> >> not
> >>> >>> >>> >>>> >>>>> >> the
> >>> >>> >>> >>>> >>>>> >> best
> >>> >>> >>> >>>> >>>>> >> solution but I at least solve temporally these
> >>> >>> >>> >>>> >>>>> >> problems:
> >>> >>> >>> >>>> >>>>> >>
> >>> >>> >>> >>>> >>>>> >> argv problem:
> >>> >>> >>> >>>> >>>>> >>
> >>> >>> >>> >>>> >>>>> >> Sentence: 'from yt.mods import *'
> >>> >>> >>> >>>> >>>>> >> Error: for any reason paraview's python shell has
> >>> >>> >>> >>>> >>>>> >> not
> >>> >>> >>> >>>> >>>>> >> defined
> >>> >>> >>> >>>> >>>>> >> the
> >>> >>> >>> >>>> >>>>> >> variable
> >>> >>> >>> >>>> >>>>> >> 'sys.argv'
> >>> >>> >>> >>>> >>>>> >> Temporal solution: comment yt code where the
> >>> >>> >>> >>>> >>>>> >> sys.argv
> >>> >>> >>> >>>> >>>>> >> variable
> >>> >>> >>> >>>> >>>>> >> is used:
> >>> >>> >>> >>>> >>>>> >>
> >>> >>> >>> >>>> >>>>> >> yt/funcs.py: lines 439-453
> >>> >>> >>> >>>> >>>>> >>
> >>> >>> >>> >>>> >>>>> >>
> >>> >>> >>> >>>> >>>>> >>
> yt/utilities/parallel_tools/parallel_analysis_interface.py:
> >>> >>> >>> >>>> >>>>> >> lines 47
> >>> >>> >>> >>>> >>>>> >> and 48
> >>> >>> >>> >>>> >>>>> >
> >>> >>> >>> >>>> >>>>> > We can likely get around this on the yt side,
> >>> >>> >>> >>>> >>>>> > although
> >>> >>> >>> >>>> >>>>> > when
> >>> >>> >>> >>>> >>>>> > embedding
> >>> >>> >>> >>>> >>>>> > yt in Enzo we've escaped this by using the command
> >>> >>> >>> >>>> >>>>> > PySys_SetArgv(argc,
> >>> >>> >>> >>>> >>>>> > argv) immediately after calling Py_Initialize.
>  Does
> >>> >>> >>> >>>> >>>>> > Partiview
> >>> >>> >>> >>>> >>>>> > report
> >>> >>> >>> >>>> >>>>> > a different executable name, by any chance?  If so
> we
> >>> >>> >>> >>>> >>>>> > can
> >>> >>> >>> >>>> >>>>> > use
> >>> >>> >>> >>>> >>>>> > that as
> >>> >>> >>> >>>> >>>>> > a signal to avoid requesting argv.
> >>> >>> >>> >>>> >>>>> >
> >>> >>> >>> >>>> >>>>> >>
> >>> >>> >>> >>>> >>>>> >> :
> >>> >>> >>> >>>> >>>>> >>
> >>> >>> >>> >>>> >>>>> >> Sentence: "pf =
> load('tests/DD0010/moving7_0010')"
> >>> >>> >>> >>>> >>>>> >> Error: python shell does not complete the load
> >>> >>> >>> >>>> >>>>> >> function
> >>> >>> >>> >>>> >>>>> >> and
> >>> >>> >>> >>>> >>>>> >> paraview
> >>> >>> >>> >>>> >>>>> >> keeps
> >>> >>> >>> >>>> >>>>> >> waiting.
> >>> >>> >>> >>>> >>>>> >> Temporal solution: When I debug paraview and
> >>> >>> >>> >>>> >>>>> >> interrupt
> >>> >>> >>> >>>> >>>>> >> its
> >>> >>> >>> >>>> >>>>> >> execution
> >>> >>> >>> >>>> >>>>> >> the
> >>> >>> >>> >>>> >>>>> >> backtrace shows sem_wait(). It happens inside the
> >>> >>> >>> >>>> >>>>> >> load
> >>> >>> >>> >>>> >>>>> >> function
> >>> >>> >>> >>>> >>>>> >> when
> >>> >>> >>> >>>> >>>>> >> tries
> >>> >>> >>> >>>> >>>>> >> to identify the candidates for input file
> >>> >>> >>> >>>> >>>>> >> (convenience.py
> >>> >>> >>> >>>> >>>>> >> line
> >>> >>> >>> >>>> >>>>> >> 74).
> >>> >>> >>> >>>> >>>>> >> There
> >>> >>> >>> >>>> >>>>> >> are 11 possible candidates and gives error for 2
> of
> >>> >>> >>> >>>> >>>>> >> them:
> >>> >>> >>> >>>> >>>>> >> ChomboStaticOutput
> >>> >>> >>> >>>> >>>>> >> and FlashStaticOutput. Skipping those candidates
> >>> >>> >>> >>>> >>>>> >> solve
> >>> >>> >>> >>>> >>>>> >> temporally the
> >>> >>> >>> >>>> >>>>> >> problem: (yt/convenienve.py line 75) inside the
> loop
> >>> >>> >>> >>>> >>>>> >> add if
> >>> >>> >>> >>>> >>>>> >> n ==
> >>> >>> >>> >>>> >>>>> >> 'ChomboStaticOutput' or n == 'FlashStaticOutput':
> >>> >>> >>> >>>> >>>>> >> continue;
> >>> >>> >>> >>>> >>>>> >
> >>> >>> >>> >>>> >>>>> > This is interesting.  What those two routines have
> in
> >>> >>> >>> >>>> >>>>> > common
> >>> >>> >>> >>>> >>>>> > is
> >>> >>> >>> >>>> >>>>> > that
> >>> >>> >>> >>>> >>>>> > they are the only two that try opening the file
> using
> >>> >>> >>> >>>> >>>>> > h5py.
> >>> >>> >>> >>>> >>>>> >  HDF5
> >>> >>> >>> >>>> >>>>> > has
> >>> >>> >>> >>>> >>>>> > a number of error handling routines, which in the
> >>> >>> >>> >>>> >>>>> > past
> >>> >>> >>> >>>> >>>>> > I've
> >>> >>> >>> >>>> >>>>> > seen
> >>> >>> >>> >>>> >>>>> > issues with when embedding one app in another --
> h5py
> >>> >>> >>> >>>> >>>>> > for
> >>> >>> >>> >>>> >>>>> > instance
> >>> >>> >>> >>>> >>>>> > handles error handling, but that can conflict with
> >>> >>> >>> >>>> >>>>> > other
> >>> >>> >>> >>>> >>>>> > mechanisms
> >>> >>> >>> >>>> >>>>> > that are attempting to handle those errors.
> >>> >>> >>> >>>> >>>>> >
> >>> >>> >>> >>>> >>>>> > Another solution that would avoid digging into the
> >>> >>> >>> >>>> >>>>> > source
> >>> >>> >>> >>>> >>>>> > code
> >>> >>> >>> >>>> >>>>> > would
> >>> >>> >>> >>>> >>>>> > be to remove the types from the actual type
> registry:
> >>> >>> >>> >>>> >>>>> >
> >>> >>> >>> >>>> >>>>> > from yt.convenience import output_type_registry
> >>> >>> >>> >>>> >>>>> > del output_type_registry["ChomboStaticOutput"]
> >>> >>> >>> >>>> >>>>> > del output_type_registry["FlashStaticOutput"]
> >>> >>> >>> >>>> >>>>> >
> >>> >>> >>> >>>> >>>>> > I'm more interested in figuring out how to fix the
> >>> >>> >>> >>>> >>>>> > interplay
> >>> >>> >>> >>>> >>>>> > between
> >>> >>> >>> >>>> >>>>> > hdf5-paraview and h5py, though.  Do you happen to
> >>> >>> >>> >>>> >>>>> > have a
> >>> >>> >>> >>>> >>>>> > full
> >>> >>> >>> >>>> >>>>> > stack
> >>> >>> >>> >>>> >>>>> > trace?
> >>> >>> >>> >>>> >>>>> >
> >>> >>> >>> >>>> >>>>> > Thanks very much for these reports -- this is very
> >>> >>> >>> >>>> >>>>> > exciting,
> >>> >>> >>> >>>> >>>>> > and
> >>> >>> >>> >>>> >>>>> > I'm
> >>> >>> >>> >>>> >>>>> > eager to help out!
> >>> >>> >>> >>>> >>>>> >
> >>> >>> >>> >>>> >>>>> > Best,
> >>> >>> >>> >>>> >>>>> >
> >>> >>> >>> >>>> >>>>> > Matt
> >>> >>> >>> >>>> >>>>> >
> >>> >>> >>> >>>> >>>>> >>
> >>> >>> >>> >>>> >>>>> >> Any of you have a better way to solve the
> problems
> >>> >>> >>> >>>> >>>>> >> above?
> >>> >>> >>> >>>> >>>>> >> Regards,
> >>> >>> >>> >>>> >>>>> >> On Wed, May 25, 2011 at 9:43 AM, Jorge Poco
> >>> >>> >>> >>>> >>>>> >> <jorge.poco at kitware.com>
> >>> >>> >>> >>>> >>>>> >> wrote:
> >>> >>> >>> >>>> >>>>> >>>
> >>> >>> >>> >>>> >>>>> >>> Hi
> >>> >>> >>> >>>> >>>>> >>> I have been trying to use YT inside the
> Paraview's
> >>> >>> >>> >>>> >>>>> >>> python
> >>> >>> >>> >>>> >>>>> >>> shell.
> >>> >>> >>> >>>> >>>>> >>> Initially
> >>> >>> >>> >>>> >>>>> >>> I have had some problems which are now
> identified.
> >>> >>> >>> >>>> >>>>> >>> It
> >>> >>> >>> >>>> >>>>> >>> is
> >>> >>> >>> >>>> >>>>> >>> not
> >>> >>> >>> >>>> >>>>> >>> the
> >>> >>> >>> >>>> >>>>> >>> smart solution but I at least solve temporally
> >>> >>> >>> >>>> >>>>> >>> these
> >>> >>> >>> >>>> >>>>> >>> problems:
> >>> >>> >>> >>>> >>>>> >>>
> >>> >>> >>> >>>> >>>>> >>> argv problem:
> >>> >>> >>> >>>> >>>>> >>>
> >>> >>> >>> >>>> >>>>> >>> Sentence: 'from yt.mods import *'
> >>> >>> >>> >>>> >>>>> >>> Error: for any reason paraview's python shell
> has
> >>> >>> >>> >>>> >>>>> >>> not
> >>> >>> >>> >>>> >>>>> >>> defined
> >>> >>> >>> >>>> >>>>> >>> the
> >>> >>> >>> >>>> >>>>> >>> variable
> >>> >>> >>> >>>> >>>>> >>> 'sys.argv'
> >>> >>> >>> >>>> >>>>> >>> Temporal solution: comment yt code where the
> >>> >>> >>> >>>> >>>>> >>> sys.argv
> >>> >>> >>> >>>> >>>>> >>> variable
> >>> >>> >>> >>>> >>>>> >>> is
> >>> >>> >>> >>>> >>>>> >>> used:
> >>> >>> >>> >>>> >>>>> >>>
> >>> >>> >>> >>>> >>>>> >>> yt/funcs.py: lines 439-453
> >>> >>> >>> >>>> >>>>> >>>
> >>> >>> >>> >>>> >>>>> >>>
> >>> >>> >>> >>>> >>>>> >>>
> yt/utilities/parallel_tools/parallel_analysis_interface.py:
> >>> >>> >>> >>>> >>>>> >>> lines 47
> >>> >>> >>> >>>> >>>>> >>> and
> >>> >>> >>> >>>> >>>>> >>> 48
> >>> >>> >>> >>>> >>>>> >>>
> >>> >>> >>> >>>> >>>>> >>> pa:
> >>> >>> >>> >>>> >>>>> >>>
> >>> >>> >>> >>>> >>>>> >>> Sentence: "pf =
> load('tests/DD0010/moving7_0010')"
> >>> >>> >>> >>>> >>>>> >>> Error: python shell does not complete the load
> >>> >>> >>> >>>> >>>>> >>> function
> >>> >>> >>> >>>> >>>>> >>> and
> >>> >>> >>> >>>> >>>>> >>> paraview
> >>> >>> >>> >>>> >>>>> >>> keeps
> >>> >>> >>> >>>> >>>>> >>> waiting.
> >>> >>> >>> >>>> >>>>> >>> Temporal solution: When I debug paraview and
> >>> >>> >>> >>>> >>>>> >>> interrupt
> >>> >>> >>> >>>> >>>>> >>> its
> >>> >>> >>> >>>> >>>>> >>> execution
> >>> >>> >>> >>>> >>>>> >>> the
> >>> >>> >>> >>>> >>>>> >>> backtrace shows sem_wait(). It happens inside
> the
> >>> >>> >>> >>>> >>>>> >>> load
> >>> >>> >>> >>>> >>>>> >>> function
> >>> >>> >>> >>>> >>>>> >>> when
> >>> >>> >>> >>>> >>>>> >>> tries
> >>> >>> >>> >>>> >>>>> >>> to identify the candidates for input file
> >>> >>> >>> >>>> >>>>> >>> (convenience.py
> >>> >>> >>> >>>> >>>>> >>> line
> >>> >>> >>> >>>> >>>>> >>> 74).
> >>> >>> >>> >>>> >>>>> >>> There
> >>> >>> >>> >>>> >>>>> >>> are 11 possible candidates and gives error for 2
> of
> >>> >>> >>> >>>> >>>>> >>> them:
> >>> >>> >>> >>>> >>>>> >>> ChomboStaticOutput
> >>> >>> >>> >>>> >>>>> >>> and FlashStaticOutput. Skipping those candidates
> >>> >>> >>> >>>> >>>>> >>> solve
> >>> >>> >>> >>>> >>>>> >>> temporally the
> >>> >>> >>> >>>> >>>>> >>> problem:
> >>> >>> >>> >>>> >>>>> >>
> >>> >>> >>> >>>> >>>>> >>
> >>> >>> >>> >>>> >>>>> >> _______________________________________________
> >>> >>> >>> >>>> >>>>> >> Amr mailing list
> >>> >>> >>> >>>> >>>>> >> Amr at public.kitware.com
> >>> >>> >>> >>>> >>>>> >>
> >>> >>> >>> >>>> >>>>> >>
> http://public.kitware.com/cgi-bin/mailman/listinfo/amr
> >>> >>> >>> >>>> >>>>> >>
> >>> >>> >>> >>>> >>>>> >>
> >>> >>> >>> >>>> >>>>> > _______________________________________________
> >>> >>> >>> >>>> >>>>> > Amr mailing list
> >>> >>> >>> >>>> >>>>> > Amr at public.kitware.com
> >>> >>> >>> >>>> >>>>> >
> >>> >>> >>> >>>> >>>>> >
> http://public.kitware.com/cgi-bin/mailman/listinfo/amr
> >>> >>> >>> >>>> >>>>> >
> >>> >>> >>> >>>> >>>>> _______________________________________________
> >>> >>> >>> >>>> >>>>> Amr mailing list
> >>> >>> >>> >>>> >>>>> Amr at public.kitware.com
> >>> >>> >>> >>>> >>>>>
> http://public.kitware.com/cgi-bin/mailman/listinfo/amr
> >>> >>> >>> >>>> >>>>
> >>> >>> >>> >>>> >>>>
> >>> >>> >>> >>>> >>>> _______________________________________________
> >>> >>> >>> >>>> >>>> Amr mailing list
> >>> >>> >>> >>>> >>>> Amr at public.kitware.com
> >>> >>> >>> >>>> >>>>
> http://public.kitware.com/cgi-bin/mailman/listinfo/amr
> >>> >>> >>> >>>> >>>>
> >>> >>> >>> >>>> >>>>
> >>> >>> >>> >>>> >>> _______________________________________________
> >>> >>> >>> >>>> >>> Amr mailing list
> >>> >>> >>> >>>> >>> Amr at public.kitware.com
> >>> >>> >>> >>>> >>>
> http://public.kitware.com/cgi-bin/mailman/listinfo/amr
> >>> >>> >>> >>>> >>>
> >>> >>> >>> >>>> >>
> >>> >>> >>> >>>> > _______________________________________________
> >>> >>> >>> >>>> > Amr mailing list
> >>> >>> >>> >>>> > Amr at public.kitware.com
> >>> >>> >>> >>>> > http://public.kitware.com/cgi-bin/mailman/listinfo/amr
> >>> >>> >>> >>>> >
> >>> >>> >>> >>>> _______________________________________________
> >>> >>> >>> >>>> Amr mailing list
> >>> >>> >>> >>>> Amr at public.kitware.com
> >>> >>> >>> >>>> http://public.kitware.com/cgi-bin/mailman/listinfo/amr
> >>> >>> >>> >>>
> >>> >>> >>> >>
> >>> >>> >>> >>
> >>> >>> >>> >
> >>> >>> >>> _______________________________________________
> >>> >>> >>> Amr mailing list
> >>> >>> >>> Amr at public.kitware.com
> >>> >>> >>> http://public.kitware.com/cgi-bin/mailman/listinfo/amr
> >>> >>> >>
> >>> >>> >>
> >>> >>> > _______________________________________________
> >>> >>> > Amr mailing list
> >>> >>> > Amr at public.kitware.com
> >>> >>> > http://public.kitware.com/cgi-bin/mailman/listinfo/amr
> >>> >>> >
> >>> >>> _______________________________________________
> >>> >>> Amr mailing list
> >>> >>> Amr at public.kitware.com
> >>> >>> http://public.kitware.com/cgi-bin/mailman/listinfo/amr
> >>> >>
> >>> >>
> >>> >
> >>
> >>
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/amr/attachments/20110613/ec2000bd/attachment-0001.htm>