[Paraview] ParaViewWeb - Issues with Loading ASCII STL file & Rendering the Output Fields

Sat Sep 26 20:43:55 EDT 2015

Dear Scott,
Should I just wait for the next patch release for the local rendering (WebGL) "Surface with edges" issue? 

I just tried the following commands to obtain the source from the GitLab and rebuilt the ParaViewWeb 4.4 with OSMesa, but it still does not resolve my stl loading issue. Am I still not getting latest 4.4 release with all the commits?.
git clone https://gitlab.kitware.com/paraview/paraview.git srccd src          git submodule update --init

On the other hand, I also tried to download and build from master .tar.gz file from the GitLab, but it seems not to have the latest submodule. I got the following error message:
CMake Error at CMakeLists.txt:625 (include):   include could not find load file:
     vtkModuleAPI

 CMake Error at CMakeLists.txt:626 (include):   include could not find load file:
     vtkModuleMacros

 CMake Error at CMake/ParaViewMacros.cmake:416 (include):   include could not find load file:
     vtkForwardingExecutable

I am not sure how to update to the latest submodule since I did not use the git to clone the source.
Thank you again!
Kai L.

     On Friday, September 25, 2015 4:28 PM, Scott Wittenburg <scott.wittenburg at kitware.com> wrote:

On Fri, Sep 25, 2015 at 3:24 PM, kai liu <liuwukai at yahoo.com> wrote:

Dear Scott:
I do not have problems setting the representation to "Surface" with color "T" in local deprecated rendering mode. However, I do get the same error message with representation set to  "Surface with edges" or "Wireframe" with color "T" in local deprecated rendering. 
But "Surface with edges" and "Wireframe" with color "T" in remote rendering are fine. 

When I click "choose palette" , select "Cool to Warm" and click apply properties for my cube data sample, it does not apply to the geometry. One thing I have noticed is that it seems to have post backs twice because the drop-down menu selection goes from "Cool to Warm" to "choose platter"  after I clicked apply properties. Could this be my sever response issue?

I think it probably is applying the color palette you choose to the geometry.  Think of the color palette as the starting point for your coloring scheme, and from there you can add and remove colors from the lookup table at will.  For this reason, we don't keep the value you choose for color palette in the drop-down box, because as soon as you adjust the lookup table, that palette name would no longer match the lookup table you're using.  Since the default color palette is "Cool to Warm", you don't notice any change when you choose that one again.  And since your data arrays have a numeric range of 0, you should just see blue.  If you change the color palette to "Red to Blue Rainbow", in your case, you will see the data become all red, though the color palette drop-down will not change. 
Also there is always  latency between user inputs and visualization. What I meant is that when users select and apply different properties to the visualization, it seems to take awhile to apply those properties. For example, color management's representation and color selections. Could the latency be due to our ubuntu sever specifications?

Some latency is expected, but I guess the question is how much latency you are seeing when changing properties and applying them.  A second to a couple of seconds is normal to apply property changes, but it could vary with the data you are visualizing. 
Could you please provide me the latest master git HTTPS clone URL ? I can try and rebuild the ParaViewWeb 4.4 with OSMesa. 

We have moved to gitlab recently, and I don't think you should need an account or anything to access the repo there.  Try this url:
https://gitlab.kitware.com/paraview/paraview.git

On the main gitlab page (https://gitlab.kitware.com/paraview/paraview) you should find links to the new development workflow documentation which can guide you through the process. 
My new question is that if we would like to start ParaViewWeb with our own default  view. For example, set the representation always to "Surface with edges", Set color always to "T" , set palette always to "Cool to Warm" and etc. Do we simply modify the HTML markup and possibly modify/add JavaScript to those files in www directory? . What would be the best way to handle this situation?

That is probably not the way I would go with it, but I think this is beyond the scope of normal mailing list support questions.  If you are interested in customizing the Web Visualizer for your own needs, you could consider a support contract.  See here for some information:
http://www.kitware.com/products/consulting.html

Hope this helps.
Cheers,Scott 
Sorry for all the questions. Thank you very much!
Kai L 

     On Friday, September 25, 2015 1:09 PM, Scott Wittenburg <scott.wittenburg at kitware.com> wrote:

 Hello Kai,
   So in doing some more testing it seems, that the problem may actually be with the webgl exporter when you set the representation to "Surface with edges".  Can you try to see if you have the problem with just "Surface"?  In that case, there is potentially a bug in the geometry exporter.
   Regarding the stl file issue, I did follow the same steps to build ParaViewWeb which you provided (just using hardware rendering rather than OSMesa) and I see the same issue you described.  So in this case, it seems like maybe some commit since the 4.4 release has fixed an issue with the stl reader.  If you are able to get the latest master and build it, that could fix your stl loading problem.
Hope this helps.
Cheers,Scott

On Fri, Sep 25, 2015 at 12:45 PM, kai liu <liuwukai at yahoo.com> wrote:

Dear Scott:
Thank you for the testings.
 1) Regarding using local rendering on the cube data file: It seems that with local rendering, we may not properly handle coloring by arrays where the data range is 0 (e.g. the range of T in your case was [303.1499938964844, 303.1499938964844]).  I was able to use local rendering by first loading the data, then choosing a local rendering approach (either worked fine), and then going back and coloring by T.  This could be a workaround if you need to test the speed of local rendering vs remote.

Kai's Response:
My local deprecated (WebGL) rendering seems quite fast now when I load ParaView Datasets. 
I was not able to use local rendering (Local VGL or Local Deprecated) on my cube data file. I first loaded the data, chose local (Deprecated) option and then went back to color by T. I got the same error message: WebSocket connection to 'ws://XXXX:8080/proxy?sessionId=7bb6e67a-63a9-11e5-9b44-00155dd73e0d' failed: Error during WebSocket handshake: Unexpected response code: 503 . I believe the server side process indeed crashed. However, remote rendering does seem to work fine with T color. The speed of local rendering is very slow, but the speed of local deprecated (WebGL) rendering is very fast now. 

2) Regarding loading the .stl file, I did not run into the same issue as you did, i.e. the .stl file loaded without any problems.  The paraviewweb I am running was built from master which is a day or so old, so I'm wondering what version you may have compiled?
Kai's Response:

I tried .stl file a couple of times again and still the same. I do not see the geometry in ParaViewWeb, and the log file has the same error message as before. Currently my Ubuntu 14.04 has ParaViewWeb - 4.4 version. I built this on 09/22/2015. The following is the commands I used to obtain the source:
Get ParaViewWeb by a Specific Version
git clone git://paraview.org/ParaView.git srccd srcgit checkout v4.4.0git submodule update --init

When I build ParaViewWeb 4.3 in July, 2015, I used the following commands:

Get the latest ParaViewWeb source 
git clone git://paraview.org/stage/ParaView.git srccd src          git submodule update --init

I have put my responses inline below again ....

On Fri, Sep 25, 2015 at 9:15 AM, kai liu <liuwukai at yahoo.com> wrote:

Dear Scott,
Thanks for your quick response, I have also put my responses inline below with light blue color :

Issue 1: Loading ASCII stl file
When I try to load my ASCII stl file on the ParaviewWeb, I receive the following error message in my log file. However, desktop ParaView application loads the file fine. 
ERROR: In /home/user/ParaViewWeb/ParaView/src/VTK/IO/Geometry/vtkSTLReader.cxx, line 461vtkSTLReader (0x5d89b50): STLReader: error while reading file /home/user/ParaViewWeb/data/Mitchell_Lee_Administration_ReckordArmory_.stl at line 14: unable to read reading point.
ERROR: In /home/user/ParaViewWeb/ParaView/src/VTK/Common/ExecutionModel/vtkExecutive.cxx, line 784vtkPVCompositeDataPipeline (0x5d84f70): Algorithm vtkFileSeriesReader(0x5d881b0) returned failure for request: vtkInformation (0x5d97f30)  Debug: Off  Modified Time: 225417  Reference Count: 1  Registered Events: (none)  Request: REQUEST_DATA  FROM_OUTPUT_PORT: 0  ALGORITHM_AFTER_FORWARD: 1  FORWARD_DIRECTION: 0

Did you build the ParaView desktop application in the same build as the version you're using to run ParaViewWeb?  Maybe you could try to take one of the standard ParaView datasets and try to load it in ParaViewWeb, and see if you have the same issue you mentioned above.  To get this data if you don't already have it, go here:
http://www.paraview.org/download/

And under "Type of Download", choose "Data, Documentation, and Tutorials".  Once you get this data, try opening the "disk_out_ref.ex2" file in ParaViewWeb.  If you see the same issue as above, then there could be something going on with the way you have built ParaView, which we can try to figure out at that point.  If you do not see the same issue as above, then perhaps you could send me (off the list) a sample data file I could use to try to replicate the problem and debug it on this end.

Kai's Response 
I did not build the ParaView desktop application. I installed ParaView 4.1.0 through the Ubuntu terminal using  sudo apt-get install paraviewopenfoam410 . 
I tested files with local ParaView desktop application as well as local ParaViewWeb. My Ubuntu 14.04 server does not have ParaView application, and it has only ParaViewWeb. Both server and local ParaViewWeb produce the same issues. 
I have tested "disk_out_ref.ex2" file in ParaViewWeb, and it seems to be okay. I will send you the sample data files privately. 

Just so I am clear.  Your "local" version is the one you install via apt-get?  And the Ubuntu 14.04 server version is the one you built yourself?  Did you download a source bundle to build the server one?  If so, do you know what version you downloaded and built on your server?
Kai's Response:

I apologize for the confusion. I built both my "local" and "sever" ParaViewWeb 4.4.0 with OSMesa using the following commands:
git clone git://paraview.org/ParaView.git srccd srcgit checkout v4.4.0git submodule update --init
I had to use git checkout v4.4.0 because last time the repository head was 4.3 version. 
My "local" ParaView 4.1 desktop application was installed via apt-get. When I said ParaView desktop application, I am not referring to the ParaViewWeb at all. I always refer ParaView and ParaViewWeb as two applications. 
Please note that I had also tried ParaViewWeb 4.1 and 4.3 on both my "local" and "server", they both had the same issues in loading the ASCII STL file and rendering the output fields, which would be the same issues currently I am experiencing in ParaViewWeb 4.4.

Issue 2: 
If I do the following selections on the ParaViewWeb, I always receive src/VTK/Common/Core/vtkDataArrayTemplate.h:191: T vtkDataArrayTemplate<T>::GetValue(vtkIdType) [with T = float; vtkIdType = long long int]: Assertion `id >= 0 && id < this->Size' failed message in the log file and cause the web socket connection closed.  

2015-09-23 17:33:52-0400 [-] Log opened.2015-09-23 17:33:53-0400 [-] Site starting on 90092015-09-23 17:33:53-0400 [-] Starting factory <twisted.web.server.Site instance at 0x7fb6f2e55830>2015-09-23 17:33:53-0400 [HTTPChannel,0,127.0.0.1] Client has reconnected, cancelling reaper2015-09-23 17:33:53-0400 [HTTPChannel,0,127.0.0.1] on_connect: connection count = 1pvpython: /home/user/ParaViewWeb/ParaView/src/VTK/Common/Core/vtkDataArrayTemplate.h:191: T vtkDataArrayTemplate<T>::GetValue(vtkIdType) [with T = float; vtkIdType = long long int]: Assertion `id >= 0 && id < this->Size' failed.
----------------------------------------------------------------------------------------------------------------------------------------------------------------
WebSocket connection to 'ws://XXXXXXXXXXX:8080/proxy?sessionId=0c562ad6-623f-11e5-9b44-00155dd73e0d' failed: Error during WebSocket handshake: Unexpected response code: 503

I checked HTTP traffic, it has the following message. I assume the connection is closed because of the above error. 

I wonder if the server side process crashed as a result of the error you see in the log file?  That would explain the websocket connection error.  Again, trying a known data set like "disk_out_ref.ex2" may help us to pinpoint problem which caused the error output you saw above.

Kai's Response 
My unintelligent question is how could I determine if the server side process crashed? 

One approach: Use a terminal on the machine where ParaViewWeb is running and type:

ps -ef | grep pvpython
If the process is running, you should see a pvpython instance running, where your pv_web_visualizer.py script is the argument, along with any other arguments your launcher provided when it started the process for you.  Something like:
/path/to/paraview/build/bin/pvpython /path/to/paraview/build/lib/site-packages/paraview/web/pv_web_visualizer.py --port 9999 --data-dir /blah/blah/blah
In the case of the cube you shared with me, the process did indeed crash as a result of the steps you provided.
Kai's Response:

Thank you for the command and the explanations.

Issue 3: Slow ParaViewWeb
My ParaViewWeb is very slow. Even I change the rendering mode to WebGL, It is still very slow. The Ubuntu 14.04 Sever  specifications are the following:
- 8 Cores - 8 GB RAM  - 2.3 Ghz processors

To get faster rendering with OSMesa, you may try building a recent version of OSMesa using Gallium + LLVMPipe.  See here:
http://www.paraview.org/Wiki/ParaView/ParaView_And_Mesa_3D

But even that won't be as fast as hardware rendering (not using OSMesa).  I'm not sure why you don't see a speedup when switching to WebGL, though, for if your data is small enough to all fit in within the GPU of your desktop or laptop, then it should be faster than offscreen rendering on the server, once the data is loaded.
Kai's Response 

I believe my build is based on  recent version of OSMesa using Gallium + LLVMPipe
Mesa 9.2.2 OSMesa Gallium llvmpipe

make -j4
autoreconf -fi
./configure \CXXFLAGS="-O2 -g -DDEFAULT_SOFTWARE_DEPTH_BITS=31" \CFLAGS="-O2 -g -DDEFAULT_SOFTWARE_DEPTH_BITS=31" \--disable-xvmc \--disable-glx \--disable-dri \--with-dri-drivers="" \--with-gallium-drivers="swrast" \--enable-texture-float \--disable-shared-glapi \--disable-egl \--with-egl-platforms="" \--enable-gallium-osmesa \--enable-gallium-llvm=yes \--with-llvm-shared-libs \--prefix=/opt/mesa/9.2.2/llvmpipe

make -j2
make -j4 install

This looks fine to me, though I believe OSMesa Gallium llvmpipe has moved along quite a few versions since that documentation was written, so building a more recent one could be something to try. 

When I built ParaViewWeb, The following is what I changed when I generated the cmake file:
PARAVIEW_ENABLE_PYTHON      ONPARAVIEW_BUILD_QT_GUI       OFFCMAKE_INSTALL_PREFIX        /.../ParaView/installVTK_USE_X                   OFFOPENGL_INCLUDE_DIR          /opt/mesa/9.2.2/llvmpipe/includeOPENGL_gl_LIBRARYOPENGL_glu_LIBRARY          /opt/mesa/9.2.2/llvmpipe/lib/libGLU.so
...

[Message clipped]  

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview/attachments/20150927/6f00a846/attachment.html>