[Paraview] Compiling Paraview on cluster

Chourasia, Amit amit at sdsc.edu
Thu Nov 3 17:09:55 EDT 2011 

David and Burlen,

Yes, the environment variables needs to be passed throught -D (doh!).
The build completed after changing to  a different MPI, I have another build compiling with boost libraries which seems to be chugging along well.

I tried `mpirun –np 4 pvserver` which seems to be listening on 11111 port

Now
1) How do I go about setting things up such that end users can use this build.
2) How do I integrate the schedulers queue for easy usage

Thanks
--Amit







On Nov 3, 2011, at 11:14 AM, David Partyka wrote:

I agree with Burlen, make sure you're passing in everything as -D arguments. Also I would start with a clean build if the mapreduce error wont go away.

On Thu, Nov 3, 2011 at 12:13 PM, Burlen Loring <bloring at lbl.gov<mailto:bloring at lbl.gov>> wrote:
 Hi Amit,

Boost is very easy to install, it's only an un tar. Then you can point PV to it with -DBOOST_ROOT on your cmake command line.

Does cmake really check environment variables for open gl and mpi paths? It's possible however, I've always passed them on the command line. That may be your issue, you could look at the CMakeCache.txt to see if it's really pickuing those up.

Burlen


On 11/03/2011 09:05 AM, Chourasia, Amit wrote:
Mesa Build:
Using Burlen's helpful tips I have moved to another step
The cluster I was compiling on had python2.4.x which is not compatible
with some stuff in mesa7.11.

I found another build of python on the cluster which I ended up using to
solve build problems. I wish mesa provided a mechanism to point to another
python version rather than making users resort to monkeying with
environment variables and aliasing trickery


Paraview Build with OSMESA (no GPU)
I am using intel compiler and mvapich/1.2.7 as a first try

1) The cluster does not have boost libraries so I had to turn
VTK_USE_BOOST=OFF.
However PARAVIEW_USE_VISITBRIDGE fails to compile because its seems to
have no method to compile when boost libraries are not available. So I had
to turn that off as well to march ahead, in the mean time will get boost
libraries installed on the cluster

How essential are boost libraries to VTK and Paraview?


2)MPI troubles

I think there are problems on two places

During configure
__________________________________________________________________
CMake Warning (dev) at
/rmount/usr_apps/shells/cmake/share/cmake-2.8/Modules/FindMPI.cmake:81
(include):
  File /rmount/usr_apps/shells/cmake/share/cmake-2.8/Modules/FindMPI.cmake
  includes /work/achourasia/ParaView-3.10.1/CMake/GetPrerequisites.cmake
  (found via CMAKE_MODULE_PATH) which shadows

/rmount/usr_apps/shells/cmake/share/cmake-2.8/Modules/GetPrerequisites.cmak
e.
  This may cause errors later on .

  Policy CMP0017 is not set: Prefer files from the CMake module directory
  when including from there.  Run "cmake --help-policy CMP0017" for policy
  details.  Use the cmake_policy command to set the policy and suppress
this
  warning.
_________________________________________________________________


During Build
__________________________________________________________________


/work/achourasia/ParaView-3.10.1/VTK/Utilities/mrmpi/src/mapreduce.cpp(14):
 catastrophic error: could not open source file "mpi.h"
  #include "mpi.h"
__________________________________________________________________



I am stumped because I do set the path to MPI include dirs and still see
this error
Will Paraview build/work correctly with Python2.4.x? If not which version
and how to specify another version of python during configuration

Here is my configure script where I set bunch of things
Again Burlen provided me most of these to get started (Thanks).

module load cmake
export CC=icc
export CXX=icpc
export  CMAKE_INSTALL_PREFIX=/work/achourasia/paraview_build
export  VTK_OPENGL_HAS_OSMESA=ON
export  VTK_USE_X=OFF
export  OSMESA_INCLUDE_DIR=/work/achourasia/mesa_build/include
export  OPENGL_INCLUDE_DIR=/work/achourasia/mesa_build/include
export  OPENGL_gl_LIBRARY=""
export  OPENGL_glu_LIBRARY=/work/achourasia/mesa_build/lib/libGLU.so
export  OSMESA_LIBRARY=/work/achourasia/mesa_build/lib/libOSMesa32.so

export PARAVIEW_BUILD_QT_GUI=OFF
export
MPI_COMPILER=/usr/apps/compilers/intel/mvapich-gen2multirail-1.0/bin/mpicxx
export MPI_LIBRARY=/usr/apps/compilers/intel/mvapich-gen2multirail-1.0/lib
export
MPI_INCLUDE_PATH=/usr/apps/compilers/intel/mvapich-gen2multirail-1.0/includ
e
export CMAKE_C_FLAGS=" -lmpi"
export CMAKE_CXX_FLAGS=" -lmpi++"

 #-DVTK_USE_BOOST=ON \
cmake \
-DCMAKE_BUILD_TYPE=Debug \
  -DBUILD_SHARED_LIBS=ON \
  -DBUILD_TESTING=OFF \
  -DPARAVIEW_DISABLE_VTK_TESTING=ON \
  -DPARAVIEW_TESTING_WITH_PYTHON=OFF \
  -DVTK_USE_BOOST=OFF \
  -DPARAVIEW_BUILD_QT_GUI=OFF \
  -DPARAVIEW_ENABLE_PYTHON=ON \
  -DPARAVIEW_USE_MPI=ON \
  -DPARAVIEW_BUILD_PLUGIN=EyeDomeLighting \
  -DPARAVIEW_USE_VISITBRIDGE=OFF \
  -DVISIT_BUILD_READER_CGNS=OFF \
  -DVISIT_BUILD_READER_Silo=OFF \
  -DBUILD_DOCUMENTATION=ON \
  -DPARAVIEW_GENERATE_PROXY_DOCUMENTATION=ON \
  -DGENERATE_FILTERS_DOCUMENTATION=ON \
  -DDOCUMENTATION_HTML_HELP=ON \
  $*



mpich/1.2.7, mpich2/1.3.2 and openmpi/1.4.1 are other things I can also
try, but the trouble is somewhere else at the moment.

I would appreciate more insight into building Paraview.


Thanks
--Amit









On 11/1/11 2:55 PM, "Burlen Loring"<bloring at lbl.gov<mailto:bloring at lbl.gov>>  wrote:

One thing I should clarify:

When I said the client and server apps can be compiled independently, I
meant with respect to X11/and OpenGL options. You will need to be aware
that other build options are not as flexible.  For instance if you want
your users to be able to use binaries from Kitware web site, you need to
enable the same plugin options, otherwise the server could crash for
instance when opening files. This is my experience up to ver 3.10, I
hope that in the future releases PV will be more intelligent about only
using a server side object if it's present.

On 11/01/2011 12:43 PM, Burlen Loring wrote:
Hi Amit,
1.  Do I compile paraview with OSMesa or Xlib? (X is available on the
cluster, will this require X on client?). Is it better to avoid X?
The client and server are separate apps and can be compiled
independently. If you have graphics hardware on your cluster you will
want to use this and link against the accelerated open gl libraries. I
believe this requires X. If you do not have graphics hardware then you
can safely disable X on the server side and use OSMesa.
   2.  Do I compile with QT or without on cluster
The server app does not need qt, and you can safely compile it without
qt. The client app obviously needs Qt. You could provide a client with
the idea that users run the client remotely (eg using nx, or vnc) or
you could ask them to install the client on their workstation and
provide a configuration script so that they can launch jobs on your
cluster. The former seems to be easier for the users while in my
experience the latter provides better performance and usability. With
the caveat that it depends on the network connection of your users.
   3.  Which compiler and which MPI to use (we have intel and gnu
compilers, MPI, mvapich and openmpi) ?
Usually this choice is somewhat arbitrary. ParaView can compile with
issue using any of the combinations. Intel is supposed to produce
faster code than gcc and be better at optimization such as auto
vectorizing. Before deciding on mvapich or openmpi you should
determine if either has an advantage given your clusters interconnect.

The error you report is an issue in Mesa, a syntax error, probably
because they used syntax that is not supported by your version of
python. I recently encountered similar issues and simply modified the
Mesa source code to use the older more universal syntax. You/we should
probably complain about this to the Mesa developers since it is a pain
the neck!

Hope it helps
Burlen

On 11/01/2011 11:58 AM, Chourasia, Amit wrote:
Hello,

I am trying to compile paraview for general usage on one of our
clusters at SDSC.
I wanted to ask what are the common paraview installation practices
that are useful for general purpose. I would certainly need to
compile the paraview server

   1.  Do I compile paraview with OSMesa or Xlib? (X is available on
the cluster, will this require X on client?). Is it better to avoid X?
   2.  Do I compile with QT or without on cluster
   3.  Which compiler and which MPI to use (we have intel and gnu
compilers, MPI, mvapich and openmpi) ?

I looked at the following page and unsure how to go about this
http://paraview.org/Wiki/ParaView_And_Mesa_3D

At the moment I am stuck on mesa compilation and the choice I make
here will determine Paraview's compilation.
When I try to compile Mesa-7.11 or Mesa-7.9 I get the following error

python2 -t -O -O builtins/tools/generate_builtins.py
./builtin_compiler>   builtin_function.cpp || rm -f builtin_function.cpp
   File "builtins/tools/generate_builtins.py", line 28
     with open(filename) as f:
             ^
SyntaxError: invalid syntax

I have tried to use
A) make linux-x86_64
B) ./configure --with-driver=xlib
--prefix=/work/achourasia/mesa_build/ --disable-gallium
--with-gallium-drivers=""
If I don¹t pass --disable-gallium  --with-gallium-drivers="" the
configure fails
checking for llvm-config... no
configure: error: LLVM is required to build Gallium R300 on x86 and
x86_64

Please advice
Thanks
--Amit





_______________________________________________
Powered by www.kitware.com<http://www.kitware.com/>

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at:
http://paraview.org/Wiki/ParaView

Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview
_______________________________________________
Powered by www.kitware.com<http://www.kitware.com/>

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView

Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview

_______________________________________________
Powered by www.kitware.com<http://www.kitware.com/>

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView

Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview

More information about the ParaView mailing list