[Paraview] Building on Titan using ParaViewSuperbuild

Fri Aug 30 17:33:33 EDT 2013

Hi Hong,
I was able to get David's instructions to work using CMake 2.8.11.2 and ParaView 4.0.1. The build process seems to be sensitive to versions.
Mark

From: Hong Yi [mailto:hongyi at renci.org]
Sent: Friday, August 30, 2013 3:52 PM
To: David E DeMarle; Vanmoer, Mark W
Cc: paraview at paraview.org
Subject: RE: [Paraview] Building on Titan using ParaViewSuperbuild

I tried to follow the instructions and the configuration scripts to build ParaView for compute nodes on Titan. It built successfully without issues for the TOOLS stage, but when doing final linking for paraview in the CROSS stage, I got numerous similar linking error from different lines such as the following:

----------------
/....../ParaView/ParaViewSuperbuild/cross/paraview/src/paraview/Utilities/mpi4py/src/mpi4py.MPI.c: In function '__pyx_pf_6mpi4py_3MPI_4File_54Sync':
/....../ParaView/ParaViewSuperbuild/cross/paraview/src/paraview/Utilities/mpi4py/src/mpi4py.MPI.c:89682:7: error: '_save' undeclared (first use in this function)
----------------

The same error message were raised from the same file mpi4py.MPI.c from different lines.

I am using CMake version 2.8.10.2 which is provided by Titan, and using the ParaView source tree version 3.98 with one additional filter I have developed.

Any idea on what could cause the linking error?

Thanks,
Hong
________________________________
From: David E DeMarle [dave.demarle at kitware.com]
Sent: Thursday, August 29, 2013 4:08 PM
To: Vanmoer, Mark W
Cc: Hong Yi; paraview at paraview.org<mailto:paraview at paraview.org>
Subject: Re: [Paraview] Building on Titan using ParaViewSuperbuild
On Thu, Aug 29, 2013 at 3:51 PM, Vanmoer, Mark W <mvanmoer at illinois.edu<mailto:mvanmoer at illinois.edu>> wrote:
So coprocessing will not be built using the below instructions? I would have mentioned that, but coprocessing appears to
still be part of a regular, non-cross-compile build, so I figured it was part of ENABLE_paraview

The coprocessing plugin, which adds things to the GUI to make it easy to record coprocessing pipeline setups doesn't need to be turned on since that lives in the client only. (It is like python trace or state recording, but tailored to recording in-situ setups).

Catalyst (the stripped down version of ParaView server that a simulation code can link to and use to run those recorded pipelines quickly) is not yet an option in ParaViewSuperbuild. To cross compile Catalyst a bit more work will be required. It will follow the same plan as how the ParaView server is compiled, but I just haven't tried it. When I did cross compile Catalyst last year at this time I did the same steps that ParaViewSuperbuild's TOOLS and CROSS build passes did, just by hand.
Also, for the below configcross.sh, do we need to pass in a CMake variable telling it where the tools build dir is located?

That should be an option that you can easily set, but it isn't sorry.

 CMake/CrossCompilationMacros.cmake assumes it can find it one directory up and over like so:
macro(find_hosttools)
  set(PARAVIEW_HOSTTOOLS_DIR ${CMAKE_BINARY_DIR}/../tools/paraview/src/paraview-build/ CACHE PATH
    "Location of host built paraview compile tools directory")
  set(PYTHON_HOST_EXE ${CMAKE_BINARY_DIR}/../tools/install/bin/python CACHE PATH
    "Location of host built python executable")
  set(PYTHON_HOST_LIBDIR ${CMAKE_BINARY_DIR}/../tools/install/lib CACHE PATH
    "Location of host built python libraries")
  set(BOOST_HOST_INCLUDEDIR ${CMAKE_BINARY_DIR}/../tools/install/include CACHE PATH
    "Location of host built boost headers")
endmacro()

You could predefine all four of those if you like.

Thanks,
Mark

From: David E DeMarle [mailto:dave.demarle at kitware.com<mailto:dave.demarle at kitware.com>]
Sent: Thursday, August 29, 2013 1:41 PM
To: Hong Yi
Cc: Vanmoer, Mark W; paraview at paraview.org<mailto:paraview at paraview.org>

Subject: Re: [Paraview] Building on Titan using ParaViewSuperbuild

On Thu, Aug 29, 2013 at 2:13 PM, Hong Yi <hongyi at renci.org<mailto:hongyi at renci.org>> wrote:
Hi David,

I just started to try superbuild on Titan also. I don't see you set ENABLE_MPI to be true in your configure script. Could you confirm whether ENABLE_MPI needs to be set to TRUE in order for ParaView to run on Titan in parallel? Since my purpose is to link our

The ENABLE_MPI flag at the Superbuild level is unrelated. It has a purpose only when CROSS_BUILD_STAGE=HOST, that is when making ParaView binary installers for desktops from Superbuild.

You shouldn't turn it on in the TOOLS or CROSS stages. Instead let the CROSS stage use the system installed MPI. It does that by turning PARAVIEW_USE_MPI=ON when it configures the ParaView sub-build. See CMake/crosscompile/xk7_gnu to see where it does that, and to see the other flags it uses.

simulation code (already built statically with CMake on Titan) to ParaView CoProcessing libraries (I am using version 3.98.1) for in-situ visualization on Titan, so in this case, do I have to set ENABLE_paraview to true and do I need to enable OSMesa for ParaView to resort to off-screen rendering for in-situ visualization?

The CROSS stage turns on Python, Mesa and ParaView. Titan's accelerators don't really run X11, so Mesa is the only option for rendering there.

Although I can build ParaView from source on Titan login nodes, I am not able to run it on compute nodes, so I am starting to try superbuild hoping to be able to cross build ParaView libraries to run in-situ visualization on Titan.

I've cross compiled Catalyst itself before on a bluegene. I did it manually before SuperBuild existed. I will see if I can dig up my config scripts. Cross compiling Catalyst should be more or less that same thing as cross compiling ParaView, but a bit faster and easier because their is less code involved.

Thanks,
Hong
________________________________
From: paraview-bounces at paraview.org<mailto:paraview-bounces at paraview.org> [paraview-bounces at paraview.org<mailto:paraview-bounces at paraview.org>] on behalf of David E DeMarle [dave.demarle at kitware.com<mailto:dave.demarle at kitware.com>]
Sent: Thursday, August 29, 2013 1:21 PM

To: Vanmoer, Mark W
Cc: paraview at paraview.org<mailto:paraview at paraview.org>
Subject: Re: [Paraview] Building on Titan using ParaViewSuperbuild

Your tools build is pointing to the compiler wrapper that you normally would use to make code for the back end.
The tools build should just use plain old local gcc since we only build things at that point that run on the login node.

Try these setup scripts:
I source configtools.sh to set up my environment before I build the compile tools, and configcross.sh before before I cross compile ParaView.

configtools.sh
#use my own cmake, system one is too old
setenv PATH /autofs/na4_proj/csc035/demarle/pvdev/titan/cmake-build/bin:${PATH}
#switch compiler to compile code for front end
module unload PrgEnv-pgi
module load gcc
#configure settings for to build compile tools only
cmake \
  -DCROSS_BUILD_STAGE:STRING=TOOLS -Dcross_target:STRING=xk7_gnu \
  -DCMAKE_BUILD_TYPE:STRING=Release \
  -DBUILD_TESTING:BOOL=FALSE \
  -DParaView_FROM_GIT:BOOL=OFF \
  -DENABLE_paraview:BOOL=TRUE \
  -DENABLE_boost:BOOL=TRUE \
  -DENABLE_python:BOOL=TRUE \
  -DENABLE_portfwd:BOOL=FALSE \
  ../../ParaViewSuperbuild

then make

configcross.sh
#use my own cmake, system one is too old
setenv PATH /autofs/na4_proj/csc035/demarle/pvdev/titan/cmake-build/bin:${PATH}
#switch compiler to compile code for back end
module unload PrgEnv-pgi
module unload gcc
module load PrgEnv-gnu
#not sure why module load wasn't sufficient, but ended up needing to force
#cmake to choose the right compiler
setenv CC /opt/cray/xt-asyncpe/5.17/bin/cc
setenv CXX /opt/cray/xt-asyncpe/5.17/bin/CC
#configure settings to cross compile python, (mesa - implied), and paraview
cmake \
  -DCROSS_BUILD_STAGE:STRING=CROSS -Dcross_target:STRING=xk7_gnu \
  -DCMAKE_BUILD_TYPE:STRING=Release \
  -DBUILD_TESTING:BOOL=TRUE \
  -DParaView_FROM_GIT:BOOL=OFF \
  -DENABLE_paraview:BOOL=TRUE \
  -DENABLE_python:BOOL=TRUE \
  ../../ParaViewSuperbuild

then make again

David E DeMarle
Kitware, Inc.
R&D Engineer
21 Corporate Drive
Clifton Park, NY 12065-8662
Phone: 518-881-4909<tel:518-881-4909>

On Tue, Aug 27, 2013 at 4:26 PM, Vanmoer, Mark W <mvanmoer at illinois.edu<mailto:mvanmoer at illinois.edu>> wrote:
Hi, I'm trying to follow the advice on building ParaView on Titan using the ParaViewSuperbuild method from an earlier discussion in June.

When I run make in the "TOOLS" directory I get the following error:

[ 66%] Building CXX object Utilities/ProcessXML/CMakeFiles/kwProcessXML.dir/ProcessXML.cxx.o
Linking CXX executable ../../bin/vtkkwProcessXML-pv4.0
/usr/bin/ld: attempted static link of dynamic object `../../lib/libvtkCommonCore-pv4.0.so.1'
collect2: error: ld returned 1 exit status
make[6]: *** [bin/vtkkwProcessXML-pv4.0] Error 1
make[5]: *** [Utilities/ProcessXML/CMakeFiles/kwProcessXML.dir/all] Error 2
make[4]: *** [CMakeFiles/pvCompileTools.dir/rule] Error 2
make[3]: *** [pvCompileTools] Error 2
CMake Error at /ccs/home/vanmoer/builds/superbuild/tools-build/pv-paraview-build.cmake:26 (message):
  Failed!!!

make[2]: *** [paraview/src/paraview-stamp/paraview-build] Error 1

I don't see a BUILD_SHARED that I can toggle. All of the CMAKE_SHARED_LINKER_FLAGS* vars are empty, if those are related.

Any suggestions?

Thanks,
Mark

_______________________________________________
Powered by www.kitware.com<http://www.kitware.com>

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView

Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20130830/9a7d5bba/attachment-0001.htm>