Compiling ParaView3 for Cray supercomputers: Difference between revisions
| Line 66: | Line 66: | ||
==Directory structure== | ==Directory structure== | ||
Setup your directories however you'd like. Instructions on this wiki are usually given in two forms, a general form and an example form | Setup your directories however you'd like. Instructions on this wiki are usually given in two forms, a general form and an example form: | ||
<pre> | <pre> | ||
## | |||
General form Example form | General form Example form | ||
------------ ------------- | ------------ ------------- | ||
Revision as of 20:58, 17 December 2007
This wiki page is currently a work in progress.
Objectives and Overview
Our goal is to run pvbatch on Cray massively parallel processor systems. Pvbatch is ParaView's MPI-enabled batch application. It reads batch scripts written in python and distributes the work across many processors. Pvbatch will be built when you compile ParaView 3. Before you can compile ParaView 3 you must compile three support packages- CMake, OSMesa, Python. The entire process will take about three hours to complete and you should have at least one gigabyte of workspace.
These instructions are intended for Cray MPP systems running the Catamount operating system. Specifically, these instructions have been tested on the Bigben XT3 supercomputer at Pittsburgh Supercomputing Center.
Terminology
These terms are probably self explanatory, but just to clarify...
- front end node - the computer/shell you log into and work on.
- native operating system - the operating system that runs on the front end node (at PSC the OS is SuSE linux)
- native build - software that executes on the front end nodes
- compute node - the computers/processors running scientific computation
- catamount - the operating system that runs on the compute nodes
- catamount build - software that has been cross compiled to execute on the compute node
Build steps
You will log into a shell on a front end node. You will download the source code and then compile CMake, OSMesa, Python, and ParaView3. Some of these packages must be compiled twice- one native version and one cross compiled version. The steps are:
- Compile a CMake native build.
- Compile an OSMesa native build.
- Compile an OSMesa catamount build.
- Compile a Python native build.
- Compile a Python catamount build.
- Compile a ParaView 3 native build.
- Compile a ParaView 3 catamount build.
- Step 2 is optional if your front end system already has OSMesa installed.
- Step 4 is optional if your front end system already has Python installed.
Why are the native builds required?
During the ParaView build process helper binaries are compiled and executed to generate source files for future build targets. When you cross compile ParaView the helper binaries cannot execute since they are non-native to the front end node you are working on. The solution is to build a native version of ParaView first, and then tell CMake to use the native helper binaries while cross compiling.
Useful links
Some of these links may have restricted read permissions:
Compilers
The front end nodes have more than one compiler installed. We will use the PGI and GNU compilers. At Bigben, the PGI compiler is the default compiler when you log in. You can switch compilers like this:
## switch from PGI to GNU compiler module switch PrgEnv-pgi PrgEnv-gnu ## switch from GNU to PGI compiler module switch PrgEnv-gnu PrgEnv-pgi
Toolchains
When you cross compile with CMake you will input a toolchain file. You will use one toolchain file when you use the PGI compiler, and a different toolchain file when you use GNU.
For more information see the CMake/CrayXT3 page.
Directory structure
Setup your directories however you'd like. Instructions on this wiki are usually given in two forms, a general form and an example form:
## General form Example form ------------ ------------- <install-dir> ~/install <catamount-install-dir> ~/install-catamount <toolchain-dir> ~/toolchains <paraview-source-dir> ~/projects/paraview/ParaView3 <paraview-native-build-dir> ~/projects/paraview/build-native ... ...
Here is how my directory tree looks:
~/
install/
bin/
include/
lib/
install-catamount/
bin/
include/
lib/
toolchains/
projects/
cmake/
CMake/
build-native/
mesa/
mesa-native/
mesa-catamount/
python/
python-for-cmake/
build-native/
build-catamount/
paraview/
ParaView3/
build-native/
build-catamount/
Note, some of these directories will be created automatically when you extract archives or checkout code from cvs/svn. The install directories and subdirectories are created automatically when you run "make install" commands. Here is a command you could use to set up the directory tree:
cd ~/ mkdir toolchains projects projects/cmake projects/mesa projects/python projects/paraview
Compiling CMake
Getting the source
You will need the latest version of CMake from CVS.
cd ~/projects/cmake cvs -d :pserver:anonymous@www.cmake.org:/cvsroot/CMake login ## respond with password: cmake cvs -d :pserver:anonymous@www.cmake.org:/cvsroot/CMake co CMake
Native build
It shouldn't matter which compiler you use to build CMake. I used the default PGI compiler.
General command:
cd <cmake-build-dir> <cmake-src-dir>/bootstrap --prefix=<native-install-dir> make make install
Example command:
cd ~/projects/cmake/build ../CMake/bootstrap --prefix=~/projects/install make make install
You should now have the executable: <native-install-dir>/bin/ccmake.
Compiling OSMesa
You will download the Mesa source code and compile the OSMesa target. OSMesa (off screen mesa) allows rendering with the OpenGL API directly into main memory instead of using system display memory. The native build is only required if your native system does not have OSMesa already installed. At Bigben, OSMesa was found at /usr/lib64/libOSMesa.so with headers in /usr/include.
Getting the source
You can download the Mesa source directly using wget. In case the url changes, here is the Mesa download page
cd ~/projects/mesa wget http://easynews.dl.sourceforge.net/sourceforge/mesa3d/MesaLib-7.0.2.tar.gz tar -zxf MesaLib-7.0.2.tar.gz
Native build
Use the PGI compiler. Since Mesa uses an in-source build you might want to copy the source dir before you start.
cd ~/projects/mesa cp -r Mesa-7.0.2 mesa-native ## edit mesa-native/configs/default ## ## replace line: INSTALL_DIR = /usr/local ## with: INSTALL_DIR = ~/install ## or: INSTALL_DIR = <native-install-dir> cd mesa-native make linux-osmesa make install
Catamount build
Use the PGI compiler. Since Mesa uses an in-source build you might want to copy the source dir before you start.
cd ~/projects/mesa cp -r Mesa-7.0.2 mesa-catamount ## edit mesa-catamount/configs/default ## ## replace line: INSTALL_DIR = /usr/local ## with: INSTALL_DIR = ~/install-catamount ## or: INSTALL_DIR = <catamount-install-dir> cd mesa-catamount make catamount-osmesa-pgi make install
Compiling Python
CMake files for building Python can be checked out from the ParaView repository. The native python build is only required if your system doesn't already have python libraries and binaries installed. On Bigben, python was located at /usr/lib64/libpython2.3.so and /usr/bin/python2.3.
Getting the source
These instructions use Python from the subversion repository. It is possible to use Python release 2.5.1 and apply a patch, more details are here.
cd ~/projects/python svn co http://svn.python.org/projects/python/trunk python-with-cmake cvs -d :pserver:anoncvs@www.paraview.org:/cvsroot/ParaView3 login ## respond with empty password cvs -d :pserver:anoncvs@www.paraview.org:/cvsroot/ParaView3 co -d python-with-cmake ParaView3/Utilities/CMakeBuildForPython
Native build
Use the GNU compiler.
Example command:
cd ~/projects/python mkdir build-native ~/install/bin/ccmake ../python-with-cmake -DCMAKE_INSTALL_PREFIX=~/install ## configure with ccmake make make install
General command:
cd <python-build-native-dir> <native-install-dir>/bin/ccmake <python-source-dir> -DCMAKE_INSTALL_PREFIX=<native-install-dir> ## configure with ccmake make make install
Catamount build
Use the GNU compiler. When configuring with CMake:
- Confirm all MODULE__*_SHARED options are off
- Turn off MODULE__pwd_ENABLE
- Turn off ENABLE_IPV6
- Turn off WITH_THREAD
General command:
cd <python-build-catamount-dir> <native-install-dir>/bin/ccmake -DCMAKE_TOOLCHAIN_FILE=~<toolchain-dir>/Toolchain-Catamount-gcc.cmake -DCMAKE_INSTALL_PREFIX=<catamount-install-dir> -C <python-source-dir>/CMake/TryRunResults-Python-catamount-gcc.cmake <python-source-dir> ## configure with ccmake make make install
Example command:
cd ~/projects/python mkdir build-catamount ~/install/bin/ccmake -DCMAKE_TOOLCHAIN_FILE=~/toolchains/Toolchain-Catamount-gcc.cmake -DCMAKE_INSTALL_PREFIX=~/install-catamount -C ../python-with-cmake/CMake/TryRunResults-Python-catamount-gcc.cmake ../python-with-cmake/ ## configure with ccmake make make install
