[Paraview] Visualizing a Cone with PVW - Connection can't be established
Dmitry Duplyakin
duplyakin at uchicago.edu
Mon Jul 21 15:59:13 EDT 2014
Continue troubleshooting...
Now we get this in response to POST:
{"updir": "/Home", "sessionManagerURL": "http://54.211.22.94:/paraview/",
"port": 9999, "host": "localhost", "sessionURL": "ws://
54.211.22.94/proxy?sessionId=369f8648-110f-11e4-b402-22000b0185a8", "id":
"369f8648-110f-11e4-b402-22000b0185a8"}
Looks right to me. Then, after that POST, it does GETs that do not look
right:
54.211.22.94 - - [21/Jul/2014:19:54:29 +0000] "GET
*/ws://localhost:9999/ws?sessionId=d32a4d8a-1110-11e4-b402-22000b0185a8*
HTTP/1.1" 404 509 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9;
rv:30.0) Gecko/20100101 Firefox/30.0"
130.202.2.217 - - [21/Jul/2014:19:54:29 +0000] "GET
/proxy?sessionId=d32a4d8a-1110-11e4-b402-22000b0185a8 HTTP/1.1" 404 509 "-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:30.0) Gecko/20100101
Firefox/30.0"
And the message:
Firefox can't establish a connection to the server at ws://
54.211.22.94/proxy?sessionId=d32a4d8a-1110-11e4-b402-22000b0185a8. return
new WebSocket(url, protocols);
What can possibly cause this? Again, complete apache and launcher configs
are in the previous email.
On Mon, Jul 21, 2014 at 2:07 PM, Dmitry Duplyakin <duplyakin at uchicago.edu>
wrote:
> This helped tremendously!
>
> Both "-f" and "${host}" now make a lot more sense.
>
> POST works fine now and we can see the Rewrite rule in action. Something
> there still needs attention and we are investigating further.
>
> Thanks for you patience and all the help that allows us to move forward.
>
>
> On Mon, Jul 21, 2014 at 12:59 PM, Sebastien Jourdain <
> sebastien.jourdain at kitware.com> wrote:
>
>> Hi Dmitry,
>>
>> you may still have a typo
>>
>> => "ws://${host}/proxy?sessionId=${id}",
>>
>> ${host} should have been replaced with a static string that match the
>> host that the web browser is using to reach apache.
>>
>> Moreover, I forgot to mention that you need to add "-f" in your set of
>> arguments. That will force some output generation to fill some buffer which
>> will allow the launcher to detect when the server is actually ready.
>>
>> I've noticed that the documentation is missing them. Don't know why...
>> Will update it at some point.
>> Although this page show that option:
>> http://www.paraview.org/ParaView3/Doc/Nightly/www/js-doc/index.html#!/guide/py_launcher
>>
>> And the value for "timeout" should be way bigger than 5. Like 25 maybe.
>>
>> Seb
>>
>>
>>
>> On Mon, Jul 21, 2014 at 10:58 AM, Dmitry Duplyakin <
>> duplyakin at uchicago.edu> wrote:
>>
>>> Thanks for this detailed description! The process is starting to make
>>> sense and we can communicate more efficiently about our configuration and
>>> problems.
>>>
>>> Here is our complete launch.json (with the right sessionURL for a config
>>> with Apache, according to your email):
>>> {
>>> "configuration": {
>>> "content": "/pv/www",
>>> "log_dir": "/pv/logs",
>>> "host": "localhost",
>>> "endpoint": "paraview",
>>> "sessionURL": "ws://${host}/proxy?sessionId=${id}",
>>> "timeout": 5,
>>> "upload_dir": "/pv/data",
>>> "fields": ["file", "host", "port", "updir"],
>>> "port": 8080,
>>> "proxy_file": "/pv/conf/proxy.conf"},
>>> "apps": {
>>> "data_prober": {"cmd": ["${python_exec}",
>>> "${python_path}/paraview/web/pv_web_data_prober.py", "--port", "${port}",
>>> "--data-dir", "${data}"], "ready_line" : "Starting factory"},
>>> "pipeline": {"cmd": ["${python_exec}",
>>> "${python_path}/paraview/web/pv_web_visualizer.py", "--port", "${port}",
>>> "--data-dir", "${data}"], "ready_line" : "Starting factory"},
>>> "loader": {"cmd": ["${python_exec}",
>>> "${python_path}/paraview/web/pv_web_file_loader.py", "--port", "${port}",
>>> "--data-dir", "${data}"], "ready_line" : "Starting factory"},
>>> "cone": {"cmd": ["${python_exec}",
>>> "/pv/src/VTK/Web/Applications/Cone/server/vtk_web_cone.py", "--port",
>>> "${port}"], "ready_line" : "Starting factory"}},
>>> "properties": {
>>> "python_path": "/pv/paraview/lib/paraview-4.1/site-packages/",
>>> "data": "/pv/data",
>>> "python_exec": "/pv/paraview/bin/pvpython"},
>>> "resources": [{"port_range": [9901, 9999], "host": "localhost"}],
>>> "sessionData": {"updir": "/Home"}
>>> }
>>>
>>> Apache host file:
>>> <VirtualHost *:80>
>>> ServerAdmin webmaster at localhost
>>> DocumentRoot /pv/www
>>> <Directory /pv/www/>
>>> Options Indexes FollowSymLinks MultiViews
>>> AllowOverride None
>>> Order allow,deny
>>> allow from all
>>> </Directory>
>>>
>>> ProxyPass /paraview http://localhost:8080/paraview
>>>
>>> RewriteEngine On
>>> RewriteMap session-to-port txt:/pv/conf/proxy.conf
>>> RewriteCond %{QUERY_STRING} ^sessionId=(.*)$ [NC]
>>> RewriteRule ^/proxy.*$ ws://${session-to-port:%1}/ws [P]
>>>
>>> RewriteLog "/var/log/apache2/rewrite.log"
>>> RewriteLogLevel 3
>>> ErrorLog ${APACHE_LOG_DIR}/error.log
>>> LogLevel debug
>>> CustomLog ${APACHE_LOG_DIR}/access.log combined
>>> </VirtualHost>
>>>
>>> Now when we access http://<PVW server>/apps/Cone/ in the browser,
>>> it does POST {"sessionManagerURL":"http://<CORRECT IP FOR PVW
>>> server>:/paraview/","application":"cone"}. Then Launcher starts a process
>>> listening to a port from the range (e.g. 9999). We can see that process:
>>> ps auwx | grep cone
>>> root 27943 2.3 1.8 1015600 137664 pts/2 Sl+ 16:32 0:00
>>> /pv/paraview/lib/paraview-4.1/pvpython
>>> /pv/src/VTK/Web/Applications/Cone/server/vtk_web_cone.py --port 9999
>>>
>>> After that we get a session timeout and this response: "Session did not
>>> start before timeout expired. Check session logs."
>>>
>>> Any further advice?
>>>
>>>
>>> On Mon, Jul 21, 2014 at 10:02 AM, Sebastien Jourdain <
>>> sebastien.jourdain at kitware.com> wrote:
>>>
>>>> Hi Dmitry,
>>>>
>>>> First of all you need to understand the role of apache here in order to
>>>> see how the configuration should be.
>>>> As you may know, you can use ParaViewWeb without Apache, but such setup
>>>> is not meant for production.
>>>>
>>>> The list below is what you get when you add Apache:
>>>> - Apache can be use to serve the static content (html, js, png, css...)
>>>> - Apache provide a single entry point host + port. This prevent
>>>> cross-domain issue or firewall limitation.
>>>> - Apache will relay the WebSocket communication to the proper
>>>> ParaViewWeb instance on the backend based on a session ID. This mean that
>>>> the ParaViewWeb process DOES NOT need to be on the same machine as the
>>>> Apache Web Server.
>>>>
>>>> Then you need to understand how the launcher works and its role.
>>>> In ParaViewWeb a launcher is used to start a new visualization process
>>>> for a given user. This simply mean running a command line with some dynamic
>>>> parameters like the port number on which this specific instance should
>>>> listen.
>>>> So when a user connect to the system and request a new visualization
>>>> session, it is the responsibility of the launcher to start the process and
>>>> let the client know on which WebSocket URL it should connect to in order to
>>>> reach the requested visualization session.
>>>>
>>>> In a setup that DOES NOT contain Apache, we can not use the same port
>>>> as the main server (the process would not start). Hence, we start it with
>>>> some port and we let the user know that it should connect to
>>>> "ws://{host}:{port}/ws" where host and port get replaced with the set that
>>>> was used to create the process.
>>>>
>>>> See section "resources" in launcher configuration.
>>>>
>>>> BUT with Apache that is different, now we have a process which is meant
>>>> to relay the communication to the backend using the port and host that
>>>> Apache is using. So the launcher should let the client know what URL they
>>>> should use so Apache could handle it and forward it to the backend. Hence
>>>> the ws://{apache_host}/proxy?sessionId=${id}.
>>>> In the same time to keep track on which host/port a process is running
>>>> the launcher generate a text file with the proper mapping.
>>>> That mapping file is then used inside Apache to figure out how it
>>>> should establish the connection to the backend.
>>>>
>>>> This should help you understand how to fix your configuration.
>>>> Moreover, here is the remaining set of questions:
>>>>
>>>> > can we specifically log and debug Rewrite rule actions?
>>>>
>>>> Don't know, Google will be your friend here...
>>>>
>>>> > do we actually need ProxyPassReverse in Apache?
>>>>
>>>> Don't know, Google will be your friend here as well as the
>>>> documentation on how to setup ParaViewWeb on Ubuntu 14LTS.
>>>>
>>>> > what particular modules in Apache are required?
>>>>
>>>> Read the documentation, the list is minimum and provided in it.
>>>>
>>>> > in launch.json there are two particular lines host/resources:
>>>>
>>>> So the "host" is to select the proper network card when you have
>>>> several for the actual launcher server.
>>>>
>>>> Then the resources are use for process management and URL generation
>>>> for the client. Which mean, what name should be used by Apache to connect
>>>> to the machine that actually run a ParaViewWeb process.
>>>>
>>>> Seb
>>>>
>>>>
>>>> On Mon, Jul 21, 2014 at 8:01 AM, Dmitry Duplyakin <
>>>> duplyakin at uchicago.edu> wrote:
>>>>
>>>>> Thank you very much, Sebastien!
>>>>>
>>>>> Launcher side:
>>>>>>
>>>>>> "sessionURL": "ws://YOUR_HOST_NAME_TO_REPLACE(the one used to connect to apache)/proxy?sessionId=${id}",
>>>>>>
>>>>>> "proxy_file": "/data/proxy.txt"
>>>>>>
>>>>>> This is exactly what we have in our launcher:
>>>>>
>>>>> "sessionURL": "ws://${host}:${port}/ws",
>>>>>
>>>>> "host": "localhost",
>>>>>
>>>>> "port": 8080,
>>>>>
>>>>> Your sessionURL looks different. Do you actually use "proxy?sessionId="
>>>>> there? Ours came directly from the script you've suggested earlier:
>>>>> http://www.paraview.org/gitweb?p=ParaViewSuperbuild.git;a=blob_plain;f=Scripts/pvw-setup.py;hb=HEAD
>>>>>
>>>>>
>>>>>> Apache side
>>>>>>
>>>>>> RewriteMap session-to-port txt:/data/proxy.txt
>>>>>>
>>>>>> In Apache we have:
>>>>> RewriteEngine On
>>>>> RewriteMap session-to-port txt:/pv/conf/proxy.conf
>>>>> RewriteCond %{QUERY_STRING} ^sessionId=(.*)$ [NC]
>>>>> RewriteRule ^/proxy.*$ ws://${session-to-port:%1}/ws [P]
>>>>>
>>>>>
>>>>>> Then when a session start you can check the content of that
>>>>>> /data/proxy.txt and that should one line by running session with
>>>>>> {sessionId} localhost:yyyyy
>>>>>>
>>>>>
>>>>> Verified that those entires are added to the file and have the right
>>>>> format.
>>>>>
>>>>>
>>>>>> On the other hand you forgot that line in the config of your cone
>>>>>> app: "ready_line" : "Starting factory"
>>>>>>
>>>>>> Thanks! Added it and now the app looks like this:
>>>>>
>>>>> "cone": {"cmd": ["${python_exec}",
>>>>> "/pv/src/VTK/Web/Applications/Cone/server/vtk_web_cone.py", "--port",
>>>>> "${port}"], "ready_line" : "Starting factory"}},
>>>>>
>>>>>
>>>>>> You can check is any pvpython process is running, on which port, see
>>>>>> if that match in the proxy file (/data/proxy.txt) and also looking at the
>>>>>> log directory with the output of the pvpython process.
>>>>>>
>>>>>> A common error could also be that the apache does not have the right
>>>>>> to read the /data/proxy.txt file or that the launcher is not able to write
>>>>>> it.
>>>>>>
>>>>>
>>>>> Verified that launcher can write it and that entries are added.
>>>>>
>>>>> These are the rest of our questions (we need to sort these out because
>>>>> they have caused some confusion and we want to clear things up and do not
>>>>> come back to these issues anymore):
>>>>>
>>>>> - can we specifically log and debug Rewrite rule actions?
>>>>> - do we actually need ProxyPassReverse in Apache?
>>>>> - what particular modules in Apache are required? So far we have:
>>>>> proxy_module, proxy_http_module, and rewrite_module. We are unsure whether
>>>>> we need proxy_wstunnel_module or not.
>>>>> - in launch.json there are two particular lines:
>>>>> "host": "localhost",
>>>>> "resources": [{"port_range": [9901, 9999], "host": "localhost"}],
>>>>> where we have some confusion about using "localhost" vs. "0.0.0.0".
>>>>> What exactly should we use in either case?
>>>>>
>>>>> Appreciate your help!
>>>>>
>>>>>
>>>>>> Good luck,
>>>>>>
>>>>>> Seb
>>>>>>
>>>>>>
>>>>>> On Sun, Jul 20, 2014 at 10:12 AM, Dmitry Duplyakin <
>>>>>> duplyakin at uchicago.edu> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> My name is Dmitry Duplyakin. I am a PhD student helping to build a
>>>>>>> ParaViewWeb system at University of Chicago.
>>>>>>>
>>>>>>> We are building ParaView 4.1.0 (with OSMesa) on Amazon EC2 Ubuntu
>>>>>>> 12.04 instances.
>>>>>>>
>>>>>>> We have verified that PV builds without errors and that PVW launcher
>>>>>>> is functioning (new entries are added to the proxy file, vis sessions are
>>>>>>> created and are listening to ports in the specified range, etc.)
>>>>>>>
>>>>>>> We are trying to visualize a simple cone:
>>>>>>> (added app in launch.json)
>>>>>>> "cone": {"cmd": ["${python_exec}",
>>>>>>> "/pv/src/VTK/Web/Applications/Cone/server/vtk_web_cone.py", "--port",
>>>>>>> "${port}"]}},
>>>>>>>
>>>>>>> We have verified that vtk_web_cone.py actually gets called. In order
>>>>>>> to test it, we used /pv/www/apps/TestApp/index.html as a template and
>>>>>>> created /pv/www/apps/Cone/index.html where we have:
>>>>>>> ...
>>>>>>> var config = {
>>>>>>> sessionManagerURL: vtkWeb.properties.sessionManagerURL,
>>>>>>> application: "cone"
>>>>>>> };
>>>>>>> ...
>>>>>>>
>>>>>>> Now when we access this app via a browser at: http://<hostname>/apps/Cone/
>>>>>>> we either get a timeout or "connection can't be established" message.
>>>>>>>
>>>>>>> In the apache error log we see:
>>>>>>> [Sun Jul 20 16:04:15 2014] [error] [client 128.135.188.231] proxy:
>>>>>>> Error reading from remote server returned by /paraview/, referer:
>>>>>>> http://54.211.22.94/apps/Cone/
>>>>>>> [Sun Jul 20 16:04:15 2014] [error] [client 128.135.188.231] File
>>>>>>> does not exist: /pv/www/ws
>>>>>>>
>>>>>>> It appears that it has do to with Apache configuration, where we
>>>>>>> have:
>>>>>>> ProxyPass /paraview http://localhost:8080/paraview
>>>>>>> ProxyPassReverse /paraview http://localhost:8080/paraview
>>>>>>> RewriteEngine On
>>>>>>> RewriteMap session-to-port txt:/pv/conf/proxy.conf
>>>>>>> RewriteCond %{QUERY_STRING} ^sessionId=(.*)$ [NC]
>>>>>>> RewriteRule ^/proxy.*$ ws://${session-to-port:%1}/ws [P]
>>>>>>>
>>>>>>> Do these Rewrite commands look right? How can their actions be
>>>>>>> logged and verified?
>>>>>>>
>>>>>>> Do we actually need ProxyPassReverse here?
>>>>>>>
>>>>>>> Is it possible that we are missing a module or some additional
>>>>>>> configuration in Apache?
>>>>>>> # apachectl -t -D DUMP_MODULES
>>>>>>> ...
>>>>>>> proxy_module (shared)
>>>>>>> proxy_http_module (shared)
>>>>>>> proxy_wstunnel_module (shared)
>>>>>>> rewrite_module (shared)
>>>>>>> ...
>>>>>>> Syntax OK
>>>>>>>
>>>>>>> Any debugging advice?
>>>>>>>
>>>>>>> Your help will be much appreciated and will allow to move forward
>>>>>>> with our isntallation.
>>>>>>>
>>>>>>>
>>>>>>> -------------------------------------------------------------------------------
>>>>>>>
>>>>>>> Dmitry Duplyakin
>>>>>>> PhD student, CS at University of Colorado - Boulder
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Powered by www.kitware.com
>>>>>>>
>>>>>>> Visit other Kitware open-source projects at
>>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>>
>>>>>>> Please keep messages on-topic and check the ParaView Wiki at:
>>>>>>> http://paraview.org/Wiki/ParaView
>>>>>>>
>>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>>> http://public.kitware.com/mailman/listinfo/paraview
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview/attachments/20140721/8ffc4850/attachment-0001.html>
More information about the ParaView
mailing list