[Paraview] paraview - client-server
Pierre-Olivier Dallaire
pierre-olivier.dallaire at videotron.ca
Thu Jun 3 08:30:30 EDT 2010
Worked, thanks !
I tried different files : couple stls and openfoam cases.
Regards,
PO
On 2010-06-01, at 1:02 PM, burlen wrote:
> Hi,
>
> I have seen this error mostly relating to a plugin being loaded on the client side but not the server side. It looks like vtk1DTransferFunctionFilter, is part of a PointSprite plugin. You could check the tools->plugin manager dialog to see if the client and server have the same plugins loaded?
>
> It would also be helpful to know what type of file have you tried to open?
>
> good luck
> Burlen
>
>
>
> Pierre-Olivier Dallaire wrote:
>> Hi,
>>
>> I'm facing an error under the client-server mode. As soon as I try to open a file, paraview crashes with this error on the
>> server side :
>>
>> ERROR: In /home/podallaire/ParaView/Servers/Common/vtkProcessModule.cxx, line 1065
>> vtkProcessModule (0x851da0): Cannot create object of type "vtk1DTransferFunctionFilter".
>> while processing
>> Message 0 = New
>> Argument 0 = string_value {vtk1DTransferFunctionFilter}
>> Argument 1 = id_value {387}
>>
>>
>> ERROR: In /home/podallaire/ParaView/Servers/Common/vtkProcessModule.cxx, line 1066
>> vtkProcessModule (0x851da0): Aborting execution for debugging purposes.
>>
>> ....
>>
>>
>> Any idea what is going on ? I tried different files - same result.
>>
>> Regards,
>>
>> PO
>>
>> On 2010-04-30, at 2:49 PM, burlen wrote:
>>
>>
>>> yeah, that's a typo in the bug report. Both last night and this morning I have been unable to access mantis or the the pv web site. Page never loads. I'll try later.
>>>
>>> pat marion wrote:
>>>
>>>> Hey Burlen, on the bug report page for 10283, I think you need to fix the command line you are testing with :
>>>>
>>>> $ ssh remote cmd1 && cm2
>>>>
>>>> will execute cmd1 on remote and cmd2 locally. It should be:
>>>>
>>>> $ ssh remote "cmd1 && cmd2"
>>>>
>>>> Pat
>>>>
>>>> On Fri, Apr 30, 2010 at 9:12 AM, pat marion <pat.marion at kitware.com <mailto:pat.marion at kitware.com>> wrote:
>>>>
>>>> I have applied your patch. I agree that paraview should explicity
>>>> close the child process. But... what I am pointing out is that
>>>> calling QProcess::close() does not help in this situation. What I
>>>> am saying is that, even when paraview does kill the process, any
>>>> commands run by ssh on the other side of the netpipe will be
>>>> orphaned by sshd. Are you sure you can't reproduce it?
>>>>
>>>>
>>>> $ ssh localhost sleep 1d
>>>> $ < press control-c >
>>>> $ pidof sleep
>>>> $ # sleep is still running
>>>>
>>>> Pat
>>>>
>>>>
>>>> On Fri, Apr 30, 2010 at 2:08 AM, burlen <burlen.loring at gmail.com
>>>> <mailto:burlen.loring at gmail.com>> wrote:
>>>>
>>>> Hi Pat,
>>>>
>>>> From my point of view the issue is philosophical, because
>>>> practically speaking I couldn't reproduce the orphans with out
>>>> doing something a little odd namely, ssh ... && sleep 1d.
>>>> Although the fact that a user reported suggests that it may
>>>> occur in the real world as well. The question is this: should
>>>> an application explicitly clean up resources it allocates? or
>>>> should an application rely on the user not only knowing that
>>>> there is the potential for a resource leak but also knowing
>>>> enough to do the right thing to avoid it (eg ssh -tt ...)? In
>>>> my opinion, as a matter of principle, if PV spawns a process
>>>> it should explicitly clean it up and there should be no way it
>>>> can become an orphan. In this case the fact that the orphan
>>>> can hold ports open is particularly insidious, because further
>>>> connection attempt on that port fails with no helpful error
>>>> information. Also it is not very difficult to clean up a
>>>> spawned process. What it comes down to is a little book
>>>> keeping to hang on to the qprocess handle and a few lines of
>>>> code called from pqCommandServerStartup destructor to make
>>>> certain it's cleaned up. This is from the patch I submitted
>>>> when I filed the bug report.
>>>>
>>>> + // close running process
>>>> + if (this->Process->state()==QProcess::Running)
>>>> + {
>>>> + this->Process->close();
>>>> + }
>>>> + // free the object
>>>> + delete this->Process;
>>>> + this->Process=NULL;
>>>>
>>>> I think if the cluster admins out there new which ssh options
>>>> (GatewayPorts etc) are important for ParView to work
>>>> seamlessly, then they might be willing to open them up. It's
>>>> my impression that the folks that build clusters want tools
>>>> like PV to be easy to use, but they don't necessarily know all
>>>> the in's and out's of confinguring and running PV.
>>>>
>>>> Thanks for looking at this again! The -tt option to ssh is
>>>> indeed a good find.
>>>>
>>>> Burlen
>>>>
>>>> pat marion wrote:
>>>>
>>>> Hi all!
>>>>
>>>> I'm bringing this thread back- I have learned a couple new
>>>> things...
>>>>
>>>> -----------------------
>>>> No more orphans:
>>>>
>>>> Here is an easy way to create an orphan:
>>>>
>>>> $ ssh localhost sleep 1d
>>>> $ <press control c>
>>>>
>>>> The ssh process is cleaned up, but sshd orphans the sleep
>>>> process. You can avoid this by adding '-t' to ssh:
>>>>
>>>> $ ssh -t localhost sleep 1d
>>>>
>>>> Works like a charm! But then there is another problem...
>>>> try this command from paraview (using QProcess) and it
>>>> still leaves an orphan, doh! Go back and re-read ssh's
>>>> man page and you have the solution, use '-t' twice: ssh -tt
>>>>
>>>> -------------------------
>>>> GatewayPorts and portfwd workaround:
>>>>
>>>> In this scenario we have 3 machines: workstation,
>>>> service-node, and compute-node. I want to ssh from
>>>> workstation to service-node and submit a job that will run
>>>> pvserver on compute-node. When pvserver starts on
>>>> compute-node I want it to reverse connect to service-node
>>>> and I want service-node to forward the connection to
>>>> workstation. So here I go:
>>>>
>>>> $ ssh -R11111:localhost:11111 service-node qsub
>>>> start_pvserver.sh
>>>>
>>>> Oops, the qsub command returns immediately and closes my
>>>> ssh tunnel. Let's pretend that the scheduler doesn't
>>>> provide an easy way to keep the command alive, so I have
>>>> resorted to using 'sleep 1d'. So here I go, using -tt to
>>>> prevent orphans:
>>>>
>>>> $ ssh -tt -R11111:localhost:11111 service-node "qsub
>>>> start_pvserver.sh && sleep 1d"
>>>>
>>>> Well, this will only work if GatewayPorts is enabled in
>>>> sshd_config on service-node. If GatewayPorts is not
>>>> enabled, the ssh tunnel will only accept connections from
>>>> localhost, it will not accept a connection from
>>>> compute-node. We can ask the sysadmin to enable
>>>> GatewayPorts, or we could use portfwd. You can run
>>>> portfwd on service-node to forward port 22222 to port
>>>> 11111, then have compute-node connect to
>>>> service-node:22222. So your job script would launch
>>>> pvserver like this:
>>>>
>>>> pvserver -rc -ch=service-node -sp=22222
>>>>
>>>> Problem solved! Also convenient, we can use portfwd to
>>>> replace 'sleep 1d'. So the final command, executed by
>>>> paraview client:
>>>>
>>>> ssh -tt -R 11111:localhost:11111 service-node "qsub
>>>> start_pvserver.sh && portfwd -g -c fwd.cfg"
>>>>
>>>> Where fwd.cfg contains:
>>>>
>>>> tcp { 22222 { => localhost:11111 } }
>>>>
>>>>
>>>> Hope this helps!
>>>>
>>>> Pat
>>>>
>>>> On Fri, Feb 12, 2010 at 7:06 PM, burlen
>>>> <burlen.loring at gmail.com <mailto:burlen.loring at gmail.com>
>>>> <mailto:burlen.loring at gmail.com
>>>> <mailto:burlen.loring at gmail.com>>> wrote:
>>>>
>>>>
>>>> Incidentally, this brings up an interesting point about
>>>> ParaView with client/server. It doesn't try to
>>>> clean up it's
>>>> child processes, AFAIK. For example, if you set up
>>>> this ssh
>>>> tunnel inside the ParaView GUI (e.g., using a
>>>> command instead
>>>> of a manual connection), and you cancel the
>>>> connection, it
>>>> will leave the ssh running. You have to track down
>>>> the ssh
>>>> process and kill it yourself. It's minor thing,
>>>> but it can
>>>> also prevent future connections if you don't
>>>> realize there's a
>>>> zombie ssh that kept your ports open.
>>>>
>>>> I attempted to reproduce on my kubuntu 9.10, qt 4.5.2
>>>> system, with
>>>> slightly different results, which may be qt/distro/os
>>>> specific.
>>>>
>>>> On my system as long as the process ParaView spawns
>>>> finishes on
>>>> its own there is no problem. That's usually how one
>>>> would expect
>>>> things to work out since when the client disconnects
>>>> the server
>>>> closes followed by ssh. But, you are right that PV never
>>>> explicitly kills or otherwise cleans up after the
>>>> process it
>>>> starts. So if the spawned process for some reason
>>>> doesn't finish
>>>> orphan processes are introduced.
>>>>
>>>> I was able to produce orphan ssh processes, giving the
>>>> PV client a
>>>> server start up command that doesn't finish. eg
>>>>
>>>> ssh ... pvserver ... && sleep 100d
>>>>
>>>> I get the situation you described which prevents further
>>>> connection on the same ports. Once PV tries and fails
>>>> to connect
>>>> on th eopen ports, there is crash soon after.
>>>>
>>>> I filed a bug report with a patch:
>>>> http://www.paraview.org/Bug/view.php?id=10283
>>>>
>>>>
>>>>
>>>> Sean Ziegeler wrote:
>>>>
>>>> Most batch systems have an option to wait until the
>>>> job is
>>>> finished before the submit command returns. I know
>>>> PBS uses
>>>> "-W block=true" and that SGE and LSF have similar
>>>> options (but
>>>> I don't recall the precise flags).
>>>>
>>>> If your batch system doesn't provide that, I'd
>>>> recommend
>>>> adding some shell scripting to loop through
>>>> checking the queue
>>>> for job completion and not return until it's done.
>>>> The sleep
>>>> thing would work, but wouldn't exit when the server
>>>> finishes,
>>>> leaving the ssh tunnels (and other things like
>>>> portfwd if you
>>>> put them in your scripts) lying around.
>>>>
>>>> Incidentally, this brings up an interesting point about
>>>> ParaView with client/server. It doesn't try to
>>>> clean up it's
>>>> child processes, AFAIK. For example, if you set up
>>>> this ssh
>>>> tunnel inside the ParaView GUI (e.g., using a
>>>> command instead
>>>> of a manual connection), and you cancel the
>>>> connection, it
>>>> will leave the ssh running. You have to track down
>>>> the ssh
>>>> process and kill it yourself. It's minor thing,
>>>> but it can
>>>> also prevent future connections if you don't
>>>> realize there's a
>>>> zombie ssh that kept your ports open.
>>>>
>>>>
>>>> On 02/08/10 21:03, burlen wrote:
>>>>
>>>> I am curious to hear what Sean has to say.
>>>>
>>>> But, say the batch system returns right away
>>>> after the job
>>>> is submitted,
>>>> I think we can doctor the command so that it
>>>> will live for
>>>> a while
>>>> longer, what about something like this:
>>>>
>>>> ssh -R XXXX:localhost:YYYY remote_machine
>>>> "submit_my_job.sh && sleep
>>>> 100d"
>>>>
>>>>
>>>> pat marion wrote:
>>>>
>>>> Hey just checked out the wiki page, nice! One
>>>> question, wouldn't this
>>>> command hang up and close the tunnel after
>>>> submitting
>>>> the job?
>>>> ssh -R XXXX:localhost:YYYY remote_machine
>>>> submit_my_job.sh
>>>> Pat
>>>>
>>>> On Mon, Feb 8, 2010 at 8:12 PM, pat marion
>>>> <pat.marion at kitware.com
>>>> <mailto:pat.marion at kitware.com>
>>>> <mailto:pat.marion at kitware.com
>>>> <mailto:pat.marion at kitware.com>>
>>>> <mailto:pat.marion at kitware.com
>>>> <mailto:pat.marion at kitware.com>
>>>>
>>>> <mailto:pat.marion at kitware.com
>>>> <mailto:pat.marion at kitware.com>>>> wrote:
>>>>
>>>> Actually I didn't write the notes at the
>>>> hpc.mil <http://hpc.mil>
>>>> <http://hpc.mil> <http://hpc.mil>
>>>>
>>>> link.
>>>>
>>>> Here is something- and maybe this is the
>>>> problem that
>>>> Sean refers
>>>> to- in some cases, when I have set up a
>>>> reverse ssh
>>>> tunnel from
>>>> login node to workstation (command executed
>>>> from
>>>> workstation) then
>>>> the forward does not work when the compute node
>>>> connects to the
>>>> login node. However, if I have the compute node
>>>> connect to the
>>>> login node on port 33333, then use portfwd
>>>> to forward
>>>> that to
>>>> localhost:11111, where the ssh tunnel is
>>>> listening on
>>>> port 11111,
>>>> it works like a charm. The portfwd tricks
>>>> it into
>>>> thinking the
>>>> connection is coming from localhost and
>>>> allow the ssh
>>>> tunnel to
>>>> work. Hope that made a little sense...
>>>>
>>>> Pat
>>>>
>>>>
>>>> On Mon, Feb 8, 2010 at 6:29 PM, burlen
>>>> <burlen.loring at gmail.com
>>>> <mailto:burlen.loring at gmail.com>
>>>> <mailto:burlen.loring at gmail.com
>>>> <mailto:burlen.loring at gmail.com>>
>>>> <mailto:burlen.loring at gmail.com
>>>> <mailto:burlen.loring at gmail.com>
>>>> <mailto:burlen.loring at gmail.com
>>>> <mailto:burlen.loring at gmail.com>>>> wrote:
>>>>
>>>> Nice, thanks for the clarification. I am
>>>> guessing that
>>>> your
>>>> example should probably be the recommended
>>>> approach rather
>>>> than the portfwd method suggested on the PV
>>>> wiki. :) I
>>>> took
>>>> the initiative to add it to the Wiki. KW
>>>> let me know
>>>> if this
>>>> is not the case!
>>>>
>>>> http://paraview.org/Wiki/Reverse_connection_and_port_forwarding#Reverse_connection_over_an_ssh_tunnel
>>>>
>>>>
>>>>
>>>> Would you mind taking a look to be sure I
>>>> didn't miss
>>>> anything
>>>> or bollix it up?
>>>>
>>>> The sshd config options you mentioned may
>>>> be why your
>>>> method
>>>> doesn't work on the Pleiades system, either
>>>> that or
>>>> there is a
>>>> firewall between the front ends and compute
>>>> nodes. In
>>>> either
>>>> case I doubt the NAS sys admins are going to
>>>> reconfigure for
>>>> me :) So at least for now I'm stuck with
>>>> the two hop ssh
>>>> tunnels and interactive batch jobs. if
>>>> there were
>>>> someway to
>>>> script the ssh tunnel in my batch script I
>>>> would be
>>>> golden...
>>>>
>>>> By the way I put the details of the two hop
>>>> ssh tunnel
>>>> on the
>>>> wiki as well, and a link to Pat's hpc.mil
>>>> <http://hpc.mil>
>>>> <http://hpc.mil> <http://hpc.mil>
>>>>
>>>> notes. I don't dare try to summarize them
>>>> since I've never
>>>> used portfwd and it refuses to compile both
>>>> on my
>>>> workstation
>>>> and the cluster.
>>>>
>>>> Hopefully putting these notes on the Wiki
>>>> will save future
>>>> ParaView users some time and headaches.
>>>>
>>>>
>>>> Sean Ziegeler wrote:
>>>>
>>>> Not quite- the pvsc calls ssh with both the
>>>> tunnel options
>>>> and the commands to submit the batch job.
>>>> You don't even
>>>> need a pvsc; it just makes the interface
>>>> fancier. As long
>>>> as you or PV executes something like this
>>>> from your
>>>> machine:
>>>> ssh -R XXXX:localhost:YYYY remote_machine
>>>> submit_my_job.sh
>>>>
>>>> This means that port XXXX on remote_machine
>>>> will be the
>>>> port to which the server must connect. Port
>>>> YYYY (e.g.,
>>>> 11111) on your client machine is the one on
>>>> which PV
>>>> listens. You'd have to tell the server (in
>>>> the batch
>>>> submission script, for example) the name of
>>>> the node and
>>>> port XXXX to which to connect.
>>>>
>>>> One caveat that might be causing you
>>>> problems, port
>>>> forwarding (and "gateway ports" if the
>>>> server is running
>>>> on a different node than the login node)
>>>> must be enabled
>>>> in the remote_machine's sshd_config. If
>>>> not, no ssh
>>>> tunnels will work at all (see: man ssh and man
>>>> sshd_config). That's something that an
>>>> administrator
>>>> would need to set up for you.
>>>>
>>>> On 02/08/10 12:26, burlen wrote:
>>>>
>>>> So to be sure about what you're saying:
>>>> Your .pvsc
>>>> script ssh's to the
>>>> front end and submits a batch job which
>>>> when it's
>>>> scheduled , your batch
>>>> script creates a -R style tunnel and starts
>>>> pvserver
>>>> using PV reverse
>>>> connection. ? or are you using portfwd or a
>>>> second ssh
>>>> session to
>>>> establish the tunnel ?
>>>>
>>>> If you're doing this all from your .pvsc script
>>>> without a second ssh
>>>> session and/or portfwd that's awesome! I
>>>> haven't been
>>>> able to script
>>>> this, something about the batch system
>>>> prevents the
>>>> tunnel created
>>>> within the batch job's ssh session from
>>>> working. I
>>>> don't know if that's
>>>> particular to this system or a general fact
>>>> of life
>>>> about batch systems.
>>>>
>>>> Question: How are you creating the tunnel
>>>> in your
>>>> batch script?
>>>>
>>>> Sean Ziegeler wrote:
>>>>
>>>> Both ways will work for me in most cases,
>>>> i.e. a
>>>> "forward" connection
>>>> with ssh -L or a reverse connection with
>>>> ssh -R.
>>>>
>>>> However, I find that the reverse method is more
>>>> scriptable. You can
>>>> set up a .pvsc file that the client can
>>>> load and
>>>> will call ssh with
>>>> the appropriate options and commands for the
>>>> remote host, all from the
>>>> GUI. The client will simply wait for the
>>>> reverse
>>>> connection from the
>>>> server, whether it takes 5 seconds or 5
>>>> hours for
>>>> the server to get
>>>> through the batch queue.
>>>>
>>>> Using the forward connection method, if the
>>>> server
>>>> isn't started soon
>>>> enough, the client will attempt to connect and
>>>> then fail. I've always
>>>> had to log in separately, wait for the
>>>> server to
>>>> start running, then
>>>> tell my client to connect.
>>>>
>>>> -Sean
>>>>
>>>> On 02/06/10 12:58, burlen wrote:
>>>>
>>>> Hi Pat,
>>>>
>>>> My bad. I was looking at the PV wiki, and
>>>> thought you were talking about
>>>> doing this without an ssh tunnel and using
>>>> only port forward and
>>>> paraview's --reverse-connection option . Now
>>>> that I am reading your
>>>> hpc.mil <http://hpc.mil> <http://hpc.mil>
>>>> <http://hpc.mil> post I see
>>>>
>>>> what you
>>>> mean :)
>>>>
>>>> Burlen
>>>>
>>>>
>>>> pat marion wrote:
>>>>
>>>> Maybe I'm misunderstanding what you mean
>>>> by local firewall, but
>>>> usually as long as you can ssh from your
>>>> workstation to the login node
>>>> you can use a reverse ssh tunnel.
>>>>
>>>>
>>>> _______________________________________________
>>>> Powered by www.kitware.com
>>>> <http://www.kitware.com> <http://www.kitware.com>
>>>> <http://www.kitware.com>
>>>>
>>>> Visit other Kitware open-source projects at
>>>> http://www.kitware.com/opensource/opensource.html
>>>>
>>>> Please keep messages on-topic and check the
>>>> ParaView Wiki at:
>>>> http://paraview.org/Wiki/ParaView
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>> http://www.paraview.org/mailman/listinfo/paraview
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>> _______________________________________________
>>> Powered by www.kitware.com
>>>
>>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>>
>>> Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> http://www.paraview.org/mailman/listinfo/paraview
>>>
>>
>>
>
More information about the ParaView
mailing list