[Paraview] ParaViewWeb & TomCat
Chris Kees
cekees at gmail.com
Fri Dec 2 16:43:43 EST 2011
On Dec 2, 2011, at 2:47 PM, Sebastien Jourdain wrote:
> Hi Chris,
>
> Sorry to get back to you only now, but your question is not that trivial.
>
> So what still remains unclear for me is, how do you plan to exchange
>
> the data from your visualization pipeline on the engines and PWServer
> that is needed by ParaViewWeb ? Are you thinking of a VTK file, MPI
> stage or something else ?
>
> What I was thinking at a first thought was:
>
> - In your engine once your computation is done, you dump the data
> somewhere and you launch from your python a PWServer with some
> initialization script that will load your data so when a web client
> connects to it, everything will be setup as you expect.
>
As a first cut, I think it would be fine to have the engines write out their data. Andy can correct me if I'm wrong, but I think we could use the co-processing to write just the subset of geometry and field data that we want to display in the notebook. If I understand correctly, with that approach we could then just focus on starting PWServer, reading in the data files, and setting up the communication between tornado and tomcat.
For future reference, when we want to move beyond dumping the data, here is how each engine is connected to paraview: A module called vtkViewers has been imported by each python engine and executed the following code:
from paraview import servermanager
global globalController, newGlobalController
paraview.options.batch = True
paraview.options.symmetric = True
pm = servermanager.vtkProcessModule.GetProcessModule()
globalController = pm.GetGlobalController()
...
In addition to this bit, each engine has attached it's mesh and field data arrays to the proper VTK containers and set up a parallel visualization pipeline. It can potentially also read in a co-processing script to set up additional visualization. I was hoping that this setup would eventually allow us to start a PWServer and attach it to the pvserver which is directly connected to the VTK objects in memory on each engines.
> - On the ParaViewWeb side (tomcat web app) you will have to patch the
> code so it won't prevent you to connect to a process that was not
> started by itself. (I can help you with that, it should be pretty
> simple)
Thanks, that would be very helpful.
> - Find a way to forward the HTTP requests from tornado to tomcat
>
I'll look into that. I'm going to visit the IPython notebook developers next week and can get some help on working with tornado. I think it will help if I can get the standard ParaViewWeb setup working.
> But once again, I'm not sure I get the right picture of your setup and
> depending on how your VTK processing is done, other solution may be
> possible.
>
> Hope this provides some hints on your wondering,
>
> Seb
>
> PS: For your later mail did you get the source form here
> (http://paraview.org/ParaViewWeb.git) which is not github ?
Yes, sorry, I got it from paraview.org.
> PS2: Most of the documentation is available here:
> http://www.paraview.org/Wiki/ParaViewWeb
>
I've been working from that documentation. Somewhere on the list somebody mentioned upgrading to activemq 3.4 so I did too. I should only need activemq cpp libs, paraview, and PW server to successfully build and make tests, right? Or do I need to install and startup activemq and tomcat?
Thanks,
Chris
> On Fri, Dec 2, 2011 at 3:34 PM, Chris Kees <cekees at gmail.com> wrote:
>> Hi Seb,
>>
>> I thought I'd go ahead and try to install PWServer. I'm running on OS X 10.7
>> from the paraview and paraviewweb git hub sources updated yesterday. I'm
>> using activemq-cpp-library-3.4.0. The paraview build passed all it's test
>> and pwserver built with no complaints. I get the fulling when testing. Any
>> idea where to look for the problem? Thanks,Chris
>>
>> % make test
>> Running tests...
>> Test project /Users/cekees/ParaViewWeb-darwinports
>> Start 1: TestSimpleScenario
>> 1/3 Test #1: TestSimpleScenario ...............***Exception: SegFault 8.20
>> sec
>> Start 2: TestProxyProperties
>> 2/3 Test #2: TestProxyProperties ..............***Exception: SegFault 10.74
>> sec
>> Start 3: TestOK
>> 3/3 Test #3: TestOK ...........................***Exception: SegFault 9.86
>> sec
>>
>> 0% tests passed, 3 tests failed out of 3
>>
>> Total Test time (real) = 28.93 sec
>>
>> The following tests FAILED:
>> 1 - TestSimpleScenario (SEGFAULT)
>> 2 - TestProxyProperties (SEGFAULT)
>> 3 - TestOK (SEGFAULT)
>> Errors while running CTest
>> make: *** [test] Error 8
>> [minikees-3:~/ParaViewWeb-darwinports] cekees% more
>> Testing/Temporary/LastTest.log
>> Start testing: Dec 02 14:14 CST
>> ----------------------------------------------------------
>> 1/3 Testing: TestSimpleScenario
>> 1/3 Test: TestSimpleScenario
>> Command: "/Users/cekees/ParaViewWeb-darwinports/ParaViewAdapter/PWServer"
>> "--jms=0" "--session-identifier=TestSimpleScenario" "--log-level=ERROR"
>> "--batch-file=/Users/cekees/ParaViewWeb/ParaViewAdapter/Testing/Python/TestSimpleScenario.py"
>> Directory:
>> /Users/cekees/ParaViewWeb-darwinports/ParaViewAdapter/Testing/Python
>> "TestSimpleScenario" start time: Dec 02 14:14 CST
>> Output:
>> ----------------------------------------------------------
>> <end of output>
>> Test time = 8.19 sec
>> ----------------------------------------------------------
>> Test Failed.
>> "TestSimpleScenario" end time: Dec 02 14:15 CST
>> "TestSimpleScenario" time elapsed: 00:00:08
>>
>>
>> On Wed, Nov 30, 2011 at 11:01 AM, Andy Bauer <andy.bauer at kitware.com> wrote:
>>>
>>> Hi Chris,
>>>
>>> I think Sebastien and I need to talk a bit about this to figure it out
>>> since neither of us has enough knowledge on our own to solve this.
>>>
>>> I cc'ed the paraview mailing list since someone else may have some good
>>> input too or may want to follow your path in the future.
>>>
>>> Andy
>>>
>>> On Tue, Nov 29, 2011 at 11:39 PM, Chris Kees <cekees at gmail.com> wrote:
>>>>
>>>> Hi Sebastien and Randall,
>>>>
>>>> I think I'm probably the user Randall mentioned earlier. I'm responding
>>>> off list because these seems a bit detailed for a regular post--feel free to
>>>> post it back to the paraview list if you like.
>>>>
>>>> Here's a rough idea of what I'm up to. I'm running the IPython notebook
>>>> server (http://ipython.org/ipython-doc/dev/interactive/htmlnotebook.html).
>>>> I'm looking for a way to do interactive 3D graphics inside an IPython
>>>> "notebook". I don't claim to know all the details of how the notebook server
>>>> works but I think the set of connected services looks something like this:
>>>>
>>>> WebClient (browser) <-> Tornado Web Server <-> IPython Client <-> IPython
>>>> Kernel/Hub <-1-to-N-> Python Engines
>>>>
>>>> where <-> is just some abstract connection (e.g. http, zeromq
>>>> sockets,etc).
>>>>
>>>> This set of processes basically gets started when I start the IPython
>>>> Kernel on the command line of the server, though I can start "engines" later
>>>> as well and connect/disconnect browsers without killing anything else. The
>>>> kernel starts up a Tornado web server to begin serving up "notebook" pages,
>>>> which are JSON files containing executable python code and text "cells".
>>>> When somebody points a browser to a notebook (say
>>>> localhost:8000/some_path/test.pynb). You can then execute and run the python
>>>> code in the code cells by doing various things on the webpage representing
>>>> the notebook (push 'run cell' etc.). The code cells can be executed either
>>>> on the client python process, or the code can be sent through the hub to be
>>>> run on a collection of "engines" also running python (and communicating with
>>>> each other via MPI in my case).
>>>>
>>>> If the code cell is run on engines you can pull objects in the engines'
>>>> namespaces back onto the client. Now, on the client, if an expression
>>>> returns an object that implements the interface of an IPython DisplayObject
>>>> (http://ipython.org/ipython-doc/dev/api/generated/IPython.core.display.html),
>>>> then it gets displayed as part of the webpage in an "output" cell.
>>>> DisplayObject's can return HTML, JSON, javascirpt, SVG, and range of image
>>>> formats. I'm attaching a pdf of a simple IPython notebook that has an
>>>> expression returning a PNG DisplayObject at the very end.
>>>>
>>>> In my case, the python engines are running MPI and solve a PDE in
>>>> parallel. The data structures for the solution all get attached to VTK
>>>> objects and a visualization pipeline on the engines. Also, Andy Bauer
>>>> helped me get those VTK objects attached to paraview using the servermanager
>>>> module a few weeks ago so I can do co-processing now as well. I'm already
>>>> able to composite, render, and interact with 3D objects using my Qt front
>>>> end and the python VTK API. I think it will be fairly simple to have the
>>>> rank 0 engine generate a PNG of the rendered/composited image that will
>>>> fulfill the requirements of an IPython PNG DisplayObject. That would give us
>>>> non-interactive images of the solution in the notebook.
>>>>
>>>> My question is whether there is a way to add ParaViewWeb into this
>>>> collection of processes so we can generate not only stills but get full
>>>> interactive 3D visualization in the notebook. I was thinking that a
>>>> DisplayObject of type HTML, JSON, or Javascript might be able to write a bit
>>>> of code that would point the browser to the tomcat server running a paraview
>>>> web service. That service would itself be connected back to the same data
>>>> sets on the engines. Something like this:
>>>>
>>>> WebClient <-> Tornado <-> IPython Client .... <-> python engines (with
>>>> VTK objects and a servermanager)
>>>> ^
>>>> ^
>>>> |
>>>> |
>>>> Tomcat <-> ParaViewWS <-> PW server
>>>>
>>>> Anyway, if this seems like it's feasible I'd be willing to build all the
>>>> paraview web components on my laptop where I already have IPython, VTK, and
>>>> Paraview co-processing working. If that works then we might be able to get
>>>> some support for something more ambitious where the engines are running on
>>>> HPC machines.
>>>>
>>>> Thanks,
>>>> Chris
>>>>
>>>>
>>>> On Nov 29, 2011, at 12:26 PM, Sebastien Jourdain wrote:
>>>>
>>>> Hi Randall,
>>>>
>>>> This sounds good, so tomcat can easily be used on any user account
>>>> with no right outside his home. You can even strip down the default
>>>> package of tomcat so you get only the minimum required for ParaViewWeb
>>>> and you can run it on any port higher than 1024.
>>>>
>>>> To do that, you will need to download tomcat on your own and
>>>> unzip/untar it, then deploy ParaViewWeb into it and start tomcat
>>>> yourself.
>>>> That's basically what we are doing for its development.
>>>>
>>>> Hope this help,
>>>>
>>>> Seb
>>>>
>>>> On Tue, Nov 29, 2011 at 1:01 PM, Randall Hand <randall.hand at gmail.com>
>>>> wrote:
>>>>
>>>> Something I can run entirely without any higher-permissions to the
>>>> system,
>>>>
>>>> with no access to /etc or /usr folders.
>>>>
>>>>
>>>> Sebastien Jourdain wrote:
>>>>
>>>>
>>>> Hi Randall,
>>>>
>>>>
>>>> Can you be more specific by what you mean by "standalone" and by
>>>>
>>>> "can't use Tomcat" ?
>>>>
>>>> Because, for me tomcat can be standalone. Would you think something
>>>>
>>>> more embedded such as Jetty or something that does not involved Java
>>>>
>>>> at all ?
>>>>
>>>>
>>>> Thanks,
>>>>
>>>>
>>>> Seb
>>>>
>>>>
>>>> On Tue, Nov 29, 2011 at 10:00 AM, Randall Hand<randall.hand at gmail.com>
>>>>
>>>> wrote:
>>>>
>>>>
>>>> I have a user interested in using ParaViewWeb as a front-end to some
>>>>
>>>> visualization tasks, however we can't use TomCat in our environment. Is
>>>>
>>>> there any kind of "Standalone" service that can be used as a replacement?
>>>>
>>>>
>>>> I know several apps that expose a web interface offer a limited function
>>>>
>>>> webserver (trac, ipython, etc) that allows it to work standalone without
>>>>
>>>> any
>>>>
>>>> external webserver configured. Does such a setup exist for ParaViewWeb?
>>>>
>>>> --
>>>>
>>>> Randall Hand
>>>>
>>>> http://www.vizworld.com
>>>>
>>>>
>>>> _______________________________________________
>>>>
>>>> Powered by www.kitware.com
>>>>
>>>>
>>>> Visit other Kitware open-source projects at
>>>>
>>>> http://www.kitware.com/opensource/opensource.html
>>>>
>>>>
>>>> Please keep messages on-topic and check the ParaView Wiki at:
>>>>
>>>> http://paraview.org/Wiki/ParaView
>>>>
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>>
>>>> http://www.paraview.org/mailman/listinfo/paraview
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Randall Hand
>>>>
>>>> http://www.vizworld.com
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Powered by www.kitware.com
>>>>
>>>> Visit other Kitware open-source projects at
>>>> http://www.kitware.com/opensource/opensource.html
>>>>
>>>> Please keep messages on-topic and check the ParaView Wiki at:
>>>> http://paraview.org/Wiki/ParaView
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>> http://www.paraview.org/mailman/listinfo/paraview
>>>>
>>>>
>>>>
>>>
>>
More information about the ParaView
mailing list