[Paraview] A Forward Connection Over two ssh Tunnel

Tue Aug 19 14:56:25 EDT 2014

One thing to be aware of is the additional overhead of ssh compared to 
something like ncat or portfwd. ssh encrypts/unencrypts it's data which 
adds unnecessary overhead. I ran the 2 ssh tunnel solution at nersc for 
a while on Hopper. The performance was noticeably better with ncat which 
pipes data through untouched. The issue in large part because of the 
heavy loading and heavy network traffic on mom nodes where second tunnel 
ran. this is cray thing, you may not notice the difference between ssh 
and ncat on a typical cluster or if the system is not hammered by 1000's 
of simultaneous users as ours are. Still I think ncat is the best practice.

On 08/18/2014 08:56 PM, Utkarsh Ayachit wrote:
> Very cool! Thanks for sharing, Axel. It will be nice to link this page
> from the Wiki as a reference/example, if that's okay.
>
> Utkarsh
>
> On Mon, Aug 18, 2014 at 6:12 PM, Huebl, Axel <a.huebl at hzdr.de> wrote:
>> Dear Ehsan,
>>
>> I use a similar setup as Burlen, tunneling twice over a login node to
>> our cluster's head node.
>>
>> It also needs `GatewayPorts clientspecified` in the head node's
>> sshd_config to work with maximum comfort.
>> Feel free to check our wiki:
>>
>> https://github.com/ComputationalRadiationPhysics/picongpu/wiki/ParaView
>>
>> Best,
>> Axel
>>
>> On 18.08.2014 22:43, Burlen Loring wrote:
>>> Hi Utkarsh,
>>>
>>> Sure I can do that.
>>>
>>> Burlen
>>>
>>> On 08/18/2014 01:36 PM, Utkarsh Ayachit wrote:
>>>> Burlen,
>>>>
>>>> Very cool. When you get the chance, mind updating the Wiki?
>>>>
>>>> Thanks
>>>> Utkarsh
>>>>
>>>> On Mon, Aug 18, 2014 at 12:37 PM, Burlen Loring <bloring at lbl.gov> wrote:
>>>>> Hi Ehsan,
>>>>>
>>>>> As Utkarsh says you need to use a reverse connection. I suggest ncat
>>>>> instead
>>>>> of portfwd because portfwd is a long dead project with no support and
>>>>> it's
>>>>> failed to compile on some systems I used. Here's a schematic of the
>>>>> setup
>>>>>
>>>>> [work station]              [  login node    ]      [ compute node ]
>>>>> [  paraview  ]<-- ssh -R --<[ ncat + ncat -l ]<-----[ pvserver -rc ]
>>>>>
>>>>> in more detail:
>>>>> in your pvsc on the client
>>>>> ssh -R ${PORT}:localhost:${PORT} ${LOGIN_HOST} ...
>>>>>
>>>>> on the login node in a shell script referenced by the client's pvsc:
>>>>> let LOGIN_PORT=${PORT}+1
>>>>> ncat -l ${LOGIN_HOST} ${LOGIN_PORT} --sh-exec="ncat localhost $PORT" &
>>>>> qsub ...
>>>>>
>>>>> on the compute node in your batch script
>>>>> mpirun ... pvserver --reverse-connection --client-host ${LOGIN_HOST}
>>>>> --server-port ${LOGIN_PORT} ...
>>>>>
>>>>> You could skip ncat if you have control over the sshd config, you
>>>>> need to
>>>>> enable GatewayPorts.
>>>>>
>>>>> Burlen
>>>>>
>>>>>
>>>>> On 08/18/2014 08:18 AM, Utkarsh Ayachit wrote:
>>>>>
>>>>> It does boil down to using reverse connection and setting up recursive
>>>>> tunnels so that the pvserver and tunnel back to the client. See the
>>>>> following on the need for portfwd and how it could potentially be
>>>>> helpful here.
>>>>>
>>>>> http://www.paraview.org/Wiki/Reverse_connection_and_port_forwarding#Reverse_Connection_over_a_Reverse_ssh_Tunnel_with_portfwd
>>>>>
>>>>>
>>>>> On Mon, Aug 18, 2014 at 11:15 AM, ehsan saei <ehsan.saei at gmail.com>
>>>>> wrote:
>>>>>
>>>>> Dear Utkarsh,
>>>>> Thanks for your respond. I tried the section : " A forward connection
>>>>> over
>>>>> an ssh tunnel "
>>>>> but it doesn't work. The problem is that I have to ssh to one system and
>>>>> then ssh to the cluster again and finally I can submit a job. Do you
>>>>> have
>>>>> any other suggestions?
>>>>>
>>>>> thanks in advance,
>>>>> Ehsan
>>>>>
>>>>>
>>>>> On Mon, Aug 18, 2014 at 4:29 PM, Utkarsh Ayachit
>>>>> <utkarsh.ayachit at kitware.com> wrote:
>>>>>
>>>>> This may be of help:
>>>>> http://www.paraview.org/Wiki/Reverse_connection_and_port_forwarding
>>>>>
>>>>> Utkarsh
>>>>>
>>>>> On Mon, Aug 18, 2014 at 5:52 AM, ehsan saei <ehsan.saei at gmail.com>
>>>>> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> I'm new to paraview. I try to run pvserver on a cluster. The problem is
>>>>> that
>>>>> I need to ssh twice to get access to the cluster. To use pvserver I
>>>>> submit a
>>>>> job on the cluster and then I run pvserver using the command
>>>>> "mpirun -np 12 pvserver"
>>>>> then I pick the name and port from terminal and use it in paraview
>>>>> client.
>>>>> but paraview can't find the pvserver. Is there any way to establish a
>>>>> connection via these two ssh tunnels?
>>>>>
>>>>> Thanks in advance,
>>>>> Ehsan
>>>>>
>>>>> _______________________________________________
>>>>> Powered by www.kitware.com
>>>>>
>>>>> Visit other Kitware open-source projects at
>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>
>>>>> Please keep messages on-topic and check the ParaView Wiki at:
>>>>> http://paraview.org/Wiki/ParaView
>>>>>
>>>>> Follow this link to subscribe/unsubscribe:
>>>>> http://public.kitware.com/mailman/listinfo/paraview
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Powered by www.kitware.com
>>>>>
>>>>> Visit other Kitware open-source projects at
>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>
>>>>> Please keep messages on-topic and check the ParaView Wiki at:
>>>>> http://paraview.org/Wiki/ParaView
>>>>>
>>>>> Follow this link to subscribe/unsubscribe:
>>>>> http://public.kitware.com/mailman/listinfo/paraview
>>>>>
>>>>>
>>>> _______________________________________________
>>>> Powered by www.kitware.com
>>>>
>>>> Visit other Kitware open-source projects at
>>>> http://www.kitware.com/opensource/opensource.html
>>>>
>>>> Please keep messages on-topic and check the ParaView Wiki at:
>>>> http://paraview.org/Wiki/ParaView
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>> http://public.kitware.com/mailman/listinfo/paraview
>>> _______________________________________________
>>> Powered by www.kitware.com
>>>
>>> Visit other Kitware open-source projects at
>>> http://www.kitware.com/opensource/opensource.html
>>>
>>> Please keep messages on-topic and check the ParaView Wiki at:
>>> http://paraview.org/Wiki/ParaView
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> http://public.kitware.com/mailman/listinfo/paraview
>>>
>>>
>> --
>>
>> Axel Huebl
>> Diploma Student
>> Phone +49 351 260 3582
>> https://www.hzdr.de/crp
>> Computational Radiation Physics
>> Laser Particle Acceleration Division
>> Helmholtz-Zentrum Dresden - Rossendorf e.V.
>>
>> Bautzner Landstrasse 400, 01328 Dresden
>> POB 510119, D-01314 Dresden
>> Vorstand: Prof. Dr.Dr.h.c. R. Sauerbrey
>>            Prof. Dr.Dr.h.c. P. Joehnk
>> VR 1693 beim Amtsgericht Dresden
>>
>>
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>
>> Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/paraview
>>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/paraview