[Paraview] capability of ParaView, Catalyst in distributed computing environment ...

[Ufuk Utku Turuncoglu (BE)]

Hi,

After getting help from the list, i finished the initial implementation 
of the code but in this case i have a strange experience with Catalyst. 
The prototype code is working with allinputsgridwriter.py script and 
could write multi-piece dataset in VTK format without any problem. In 
this case, the code also handles four different input ports to get data 
in different grid structure and dimensions (2d/3d).

The main problem is that if i try to use the same code to output a png 
file after creating iso-surface from single 3d field (141x115x14 = 
227K), it is hanging. In this case, if i check the utilization of the 
processors (on Linux, Centos 7.1,

12064 turuncu   20   0 1232644 216400  77388 R 100.0  0.7  10:44.17 main.x
12068 turuncu   20   0 1672156 483712  70420 R 100.0  1.5 10:44.17 main.x
12069 turuncu   20   0 1660620 266716  70500 R 100.0  0.8 10:44.26 main.x
12070 turuncu   20   0 1660412 267204  71204 R 100.0  0.8 10:44.22 main.x
12071 turuncu   20   0 1659988 266644  71360 R 100.0  0.8 10:44.18 main.x
12065 turuncu   20   0 1220328 202224  77620 R  99.7  0.6 10:44.08 main.x
12066 turuncu   20   0 1220236 204696  77444 R  99.7  0.6 10:44.16 main.x
12067 turuncu   20   0 1219644 199116  77152 R  99.7  0.6 10:44.18 main.x
12078 turuncu   20   0 1704272 286924 102940 S  10.6  0.9 1:12.91 main.x
12074 turuncu   20   0 1704488 287668 103456 S  10.0  0.9 1:08.50 main.x
12072 turuncu   20   0 1704444 287488 103316 S   9.6  0.9 1:09.09 main.x
12076 turuncu   20   0 1704648 287268 102848 S   9.6  0.9 1:10.16 main.x
12073 turuncu   20   0 1704132 284128 103384 S   9.3  0.9 1:05.27 main.x
12077 turuncu   20   0 1706236 286228 103380 S   9.3  0.9 1:05.49 main.x
12079 turuncu   20   0 1699944 278800 102864 S   9.3  0.9 1:05.87 main.x
12075 turuncu   20   0 1704356 284408 103436 S   8.6  0.9 1:07.03 main.x

they seems normal because the co-processing component only works on a 
subset of the resource (8 processor, has utilization around 99 percent). 
The GPU utilization (from nvidia-smi command) is

+------------------------------------------------------+
| NVIDIA-SMI 352.79     Driver Version: 352.79 |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile 
Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util 
Compute M. |
|===============================+======================+======================|
|   0  Quadro K5200        Off  | 0000:42:00.0      On 
|                  Off |
| 26%   42C    P8    14W / 150W |    227MiB /  8191MiB | 0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
|  GPU       PID  Type  Process name Usage      |
|=============================================================================|
|    0      1937    G /usr/bin/Xorg                                   
81MiB |
|    0      3817    G /usr/bin/gnome-shell                           
110MiB |
|    0      9551    G paraview                                        
16MiB |
+-----------------------------------------------------------------------------+

So, the GPU is not overloaded in this case. I tested the code with two 
different version of ParaView (5.0.0 and 5.1.0). The results are same 
for both cases even if i create the co-processing Python scripts with 
same version of the ParaView that is used to compile the code. I also 
tried to use 2d field (141x115) but the result is also same and the code 
is still hanging. The different machine (MacOS+ParaView 5.0.0) works 
without problem. There might be a issue of Linux or installation but i 
am not sure and it was working before. Is there any flag or tool that 
allows to analyze Paraview deeply to find the source of the problem.

Regards,

--ufuk