<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
To close the loop on this I made a go at the superbuild.
unortunately it dies in the python build of the tools pass. This
kind of debugging effort that I'd like to avoid! especially as
compiling python is a simple matter and NERSC already has static
builds for all of the other deps. My experience with VisIt, and it's
"superbuild", has enforced the idea that such things are generally a
bad idea and make it it much harder to deploy the software. A common
issue with VisIt's build is that it works for developers on
"blessed" systems but in the wild it fails miserably. Debugging the
superbuild requires more work than making the installs directly! In
a way the superbuild becomes a replacement for documentation. I
think for 4.1 release I'm going to stay with what we have: stock PV
build, patched so that X11 deps are removed. However, I appreciate
the info about cross compiling and superbuild process!<br>
<br>
...<br>
CMake Warning:<br>
Manually-specified variables were not used by the project:<br>
<br>
CROSS_BUILD_SITE<br>
ENABLE_cgns<br>
ENABLE_ffmpeg<br>
ENABLE_hdf5<br>
ENABLE_matplotlib<br>
ENABLE_silo<br>
ENABLE_szip<br>
ENABLE_visitbridge<br>
qt_DISABLE_WEBKIT<br>
...<br>
configure: creating ./config.status<br>
config.status: creating Makefile.pre<br>
config.status: creating Modules/Setup.config<br>
config.status: creating Misc/python.pc<br>
config.status: creating Modules/ld_so_aix<br>
config.status: creating pyconfig.h<br>
configure: WARNING: unrecognized options: --enable-static<br>
creating Modules/Setup<br>
creating Modules/Setup.local<br>
creating Makefile<br>
Current working directory cannot be established.<br>
make[2]: *** [python/src/python-stamp/python-build] Error 1<br>
make[1]: *** [CMakeFiles/python.dir/all] Error 2<br>
make: *** [all] Error 2<br>
<br>
<br>
<div class="moz-cite-prefix">On 03/17/2014 02:06 PM, Burlen Loring
wrote:<br>
</div>
<blockquote cite="mid:532763DD.4090802@gmail.com" type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
nice! that settles it, I should definitely be using the
superbuild!<br>
<br>
My feeling is that it's probably still a good idea to look into
the OSMesa/libGL/X11 link issues and clean it up. But hopefully
superbuild will do the trick for our installs.<br>
<br>
Thanks<br>
Burlen<br>
<br>
<div class="moz-cite-prefix">On 03/17/2014 01:53 PM, David E
DeMarle wrote:<br>
</div>
<blockquote
cite="mid:CANjZAi8Fbf12MxpmvL=N9wOACf7DyhRF9VyqEe6cOEvAX+frkw@mail.gmail.com"
type="cite">
<div dir="ltr">Yes it is a real static build:
<div>On titan@ornl:<br>
<div>
<div>home/demarled> ldd
/sw/xk6/paraview/4.1.0/bin/pvbatch <br>
</div>
<div> not a dynamic executable</div>
<div>home/demarled> ldd
/sw/xk6/paraview/4.1.0/bin/pvserver </div>
<div> not a dynamic executable</div>
</div>
<div><br>
</div>
<div>On mira@anl:</div>
<div>
<div>[demarle@miralac1 ~]$ ldd
/soft/visualization/paraview/v4.1.0/bin/pvserver </div>
<div> not a dynamic executable</div>
<div>[demarle@miralac1 ~]$ ldd
/soft/visualization/paraview/v4.1.0/bin/pvbatch </div>
<div> not a dynamic executable</div>
</div>
<div><br>
</div>
</div>
<div>I agree there are drawbacks. First and foremost is that
XCSB is using our own python and mesa.</div>
<div><br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br clear="all">
<div>David E DeMarle<br>
Kitware, Inc.<br>
R&D Engineer<br>
21 Corporate Drive<br>
Clifton Park, NY 12065-8662<br>
Phone: 518-881-4909</div>
<br>
<br>
<div class="gmail_quote">On Mon, Mar 17, 2014 at 4:43 PM,
Burlen Loring <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:burlen.loring@gmail.com" target="_blank">burlen.loring@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> Hi Andy, <br>
<br>
do you actually get static executables out of the
superbuild (ldd pvserver reports not a dyanmic exec)?.
If so then I really should look into it. The reason I
haven't is that the superbuild adds complications and
potential failure points, which due to NERSC's
aggressive system software upgrades, I predict I will be
the one debugging. same with cross compiling. I feel
that neither superbuild nor cross compile should be
necessary on these systems, which have same cpu arch on
login and compute nodes. On the other hand it would be
worth the hassle if it is indeed producing static execs.<span
class="HOEnZb"><font color="#888888"><br>
<br>
Burlen</font></span>
<div>
<div class="h5"><br>
<br>
<div>On 03/17/2014 12:12 PM, Andy Bauer wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>Hey Burlen,<br>
<br>
</div>
The superbuild (<a moz-do-not-send="true"
href="http://paraview.org/Wiki/ParaView/Superbuild"
target="_blank">http://paraview.org/Wiki/ParaView/Superbuild</a>)
now does a pretty good job of building
statically on the Crays. Have you tried that
lately? If you look at what I did for
Garnet@ERDC's HPC center you can see how I
modified paths to compilers and libraries from
the default settings. I have a script below
that has the settings for doing the static
cross build on Garnet, in cast that helps.<br>
<br>
</div>
That being said, I'd still like the change of
not linking with X11 if using OSMesa as that
will also be useful to many people. <br>
<div><br>
</div>
<div>Regards,<br>
Andy<br>
</div>
<div><br>
================================<br>
#!/bin/bash<br>
<br>
ParaViewSuperBuildSource=/usr/local/usp/DAAC/paraview/source/ParaView-4.1.0-All/ParaViewSuperbuild<br>
cd
/usr/local/usp/DAAC/paraview/source/ParaView-4.1.0-All<br>
<br>
mkdir tools<br>
mkdir cross<br>
<br>
cd tools<br>
#switch compiler to compile code for front end<br>
#module load cmake<br>
module unload PrgEnv-pgi<br>
module unload PrgEnv-gnu<br>
#module load git<br>
module load gcc<br>
# CXX was set to CC which isn't defined unless
a PrgEnv-gnu/pgi/etc module is loaded<br>
export CXX=/opt/gcc/4.8.0/bin/g++<br>
export CC=/opt/gcc/4.8.0/bin/gcc<br>
<br>
# 2/13/2014 -- took out
-DParaView_URL_MD5:STRING=6882d488ac8a4622d8a9c632b1a57b44
\<br>
<br>
#configure settings for to build compile tools
only<br>
cmake \<br>
-DCROSS_BUILD_STAGE:STRING=TOOLS
-Dcross_target:STRING=xk7_gnu \<br>
-DCROSS_BUILD_SITE:STRING=garnet \<br>
-DCMAKE_CXX_COMPILER:FILEPATH=/opt/gcc/4.8.0/bin/g++
\<br>
-DCMAKE_C_COMPILER:FILEPATH=/opt/gcc/4.8.0/bin/gcc
\<br>
-DCMAKE_BUILD_TYPE:STRING=Release \<br>
-DBUILD_TESTING:BOOL=FALSE \<br>
-DENABLE_paraview:BOOL=TRUE \<br>
-DENABLE_boost:BOOL=TRUE \<br>
-DENABLE_python:BOOL=TRUE \<br>
-DENABLE_portfwd:BOOL=FALSE \<br>
-DParaView_FROM_GIT:BOOL=OFF \<br>
-DParaView_URL:STRING="<a
moz-do-not-send="true"
href="http://www.paraview.org/files/v4.1/ParaView-v4.1.0-source.tar.gz"
target="_blank">http://www.paraview.org/files/v4.1/ParaView-v4.1.0-source.tar.gz</a>"
\<br>
-DENABLE_visitbridge:BOOL=ON \<br>
-DENABLE_cgns=ON \<br>
-DENABLE_ffmpeg=ON \<br>
-DENABLE_hdf5=ON \<br>
-DENABLE_matplotlib=ON \<br>
-DENABLE_silo=ON \<br>
-DENABLE_szip=ON \<br>
-Dqt_DISABLE_WEBKIT=ON \<br>
${ParaViewSuperBuildSource}<br>
<br>
make -j2<br>
<br>
#switch compiler to compile code for back end<br>
# cmake and git are already loaded when I log
in<br>
cd ../cross<br>
#module load cmake<br>
module unload gcc<br>
module load PrgEnv-gnu<br>
module load gcc/4.8.0<br>
#not sure why module load wasn't sufficient,
but ended up needing to force<br>
#cmake to choose the right compiler<br>
#export CC=/opt/cray/xt-asyncpe/5.17/bin/cc<br>
#export CXX=/opt/cray/xt-asyncpe/5.17/bin/CC<br>
#export CXX=/opt/gcc/4.8.0/bin/g++<br>
#export CC=/opt/gcc/4.8.0/bin/gcc<br>
export CXX=/opt/cray/xt-asyncpe/5.21/bin/CC<br>
export CC=/opt/cray/xt-asyncpe/5.21/bin/cc<br>
#configure settings to cross compile python,
(mesa - implied), and paraview<br>
cmake \<br>
-DCROSS_BUILD_STAGE:STRING=CROSS
-Dcross_target:STRING=xk7_gnu \<br>
-DCROSS_BUILD_SITE:STRING=garnet \<br>
-DCMAKE_BUILD_TYPE:STRING=Release \<br>
-DBUILD_TESTING:BOOL=FALSE \<br>
-DENABLE_paraview:BOOL=TRUE \<br>
-DENABLE_python:BOOL=TRUE \<br>
-DParaView_FROM_GIT:BOOL=OFF \<br>
-DCMAKE_INSTALL_PREFIX:PATH=/usr/local/usp/DAAC/paraview/4.1.0_osmesa
\<br>
${ParaViewSuperBuildSource}<br>
<br>
make -j2 install<br>
<br>
# with CMAKE_INSTALL_PREFIX set, I shouldn't
have to do the manual install...<br>
# next go into
cross/paraview/src/paraview-build and set the
install directory with CMake<br>
# then do make install, copy site-packages
directory from lib/paraview-4.0 into lib.<br>
# finally pvbatch relies on where python was
installed (i.e. cross/install/lib) but<br>
# I can copy that to someplace else and set
PYTHONHOME to take care of it.<br>
# for example:<br>
# cp -r cross/install/lib/python2.7
<install>/lib<br>
# export PYTHONHOME=<install><br>
# such that $PYTHONHOME/lib/python2.7/ has all
of the default .py files<br>
==================<br>
</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Mon, Mar 17, 2014 at
3:01 PM, burlen <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:burlen.loring@gmail.com"
target="_blank">burlen.loring@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div>Thanks Guys! I appreciate your eyes
on it.<br>
<br>
As you point out clearing
OPENGL_gl_LIBRARY should prevent libGL
dependency and link order issues. In my
builds I have got into the habit of
setting it to libOSMesa which does the
same. In any case this doesn't prevent
the X11 libs from being linked. in
FindOpenGL.cmake that ships with cmake,
X11 libs are always tacked onto
OPENGL_LIBRARIES, which is used in
Rendering/OpenGL/CMakeLists. <br>
<br>
my main motivation in digging into these
issues is to address a PV performance
issue on NERSC's Crays. Users found that
PV start-ups were sometimes taking a
very long time (eg 30 min for 48 way
parallel run). We tracked the issue to
the file system and PV's use of shared
libraries. With PV there are many shared
library dependencies (~200), and when
parallel pvserver's startup they all hit
the file system at the same time all
opening these shared library files. When
I make static executables startup time
is reduced reliably to a few seconds. I
admit static executables are an extreme
measure, and I hit a handful of cmake/pv
specific cmake related issues doing it
that won't be easy to address. However,
it's probably OK performance wise to
settle for "as static as possible", and
getting the list of shared library
dependencies from ~200 to ~10 should
make for a big improvement. Removing
those X11 deps when using OSMesa gets us
"as static as possible" and without too
much effort. <br>
<span><font color="#888888"> <br>
Burlen</font></span>
<div>
<div><br>
<br>
On 03/17/2014 08:03 AM, David E
DeMarle wrote:<br>
</div>
</div>
</div>
<div>
<div>
<blockquote type="cite">
<div dir="ltr">I'll take a close
look at this too.
<div><br>
</div>
<div>Like Andy said, we do get by
for example in ParaView's cross
compiling super build, but I'm
all in for making it easier in
practice to get VTK and ParaView
up and running in HPC contexts.</div>
<div><br>
</div>
<div>Thanks as always Burlen!
<div><br>
</div>
</div>
</div>
<div class="gmail_extra"><br
clear="all">
<div>David E DeMarle<br>
Kitware, Inc.<br>
R&D Engineer<br>
21 Corporate Drive<br>
Clifton Park, NY 12065-8662<br>
Phone: <a
moz-do-not-send="true"
href="tel:518-881-4909"
value="+15188814909"
target="_blank">518-881-4909</a></div>
<br>
<br>
<div class="gmail_quote">On Mon,
Mar 17, 2014 at 10:06 AM, Marcus
D. Hanwell <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:marcus.hanwell@kitware.com"
target="_blank">marcus.hanwell@kitware.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div>On Fri, Mar 14, 2014 at
5:31 PM, Burlen Loring <<a
moz-do-not-send="true"
href="mailto:burlen.loring@gmail.com"
target="_blank">burlen.loring@gmail.com</a>>
wrote:<br>
</div>
<div>> Hi All,<br>
><br>
> While installing the
past two PV releases I've
noticed that VTK is linking<br>
> in X11 and libGL when
using OSMesa, which is
acheived by VTK_USE_X=OFF
and<br>
>
VTK_OPENGL_HAS_OSMESA=ON.
This is one a of a number of
issues that is making<br>
> statically linked PV
executables on the Cray
painful. I think that the<br>
> history of this is that
this was useful in the past
when mangled Mesa was<br>
> used but currently this
is the wrong thing to do
because the extension<br>
> manager can only use
one or the other as
GetProcAddress function is
selected<br>
> at compile time, and at
run time it's used
indiscriminatly regardless
of<br>
> actual context
type(OSMesa context or other
GL context) so there's
currently<br>
> no chance that you
could safely use both. Also,
linking both libraries can<br>
> create a link time race
condition between libGL and
libOSMesa that depends<br>
> on link order(eg both
are .so's or .a's). I think
that in a single build VTK<br>
> needs to carefully use
either libGL or OSMesa, but
not both, and that<br>
> linking against X11
should be avoided when
VTK_USE_X11=OFF. I've
tracked<br>
> the X11 library
inclusion down, it's coming
from FindOpenGL.cmake, which<br>
> appends X11 libraries
to OPENGL_LIBRARIES
indiscriminately when X11 is<br>
> found. I've pushed a
patch (which needs a careful
review and some more<br>
> testing) onto gerrit
that explicitly separates
OpenGL and OSMesa use in<br>
> VTK(<a
moz-do-not-send="true"
href="http://review.source.kitware.com/#/c/14730/"
target="_blank">http://review.source.kitware.com/#/c/14730/</a>).
I'd like to know if others<br>
> agree that this is a
reasonable change? Thoughts
and/or comments?<br>
><br>
</div>
I started looking at this
briefly before I left, and hit
the same<br>
thing Andy mentioned (having
to clear OPENGL_gl_LIBRARY to
get a<br>
working OSMesa). It would be
great to improve this part of
the build<br>
system (I wasn't aware of all
the history, but assumed it
had<br>
something to do with mangled
mesa).<br>
<br>
I will try to find some time
to take a closer look at the
proposed<br>
patch in the next few days,
and always wanted to loop back
to this<br>
logic since modularization as
it appeared overly complex but
we also<br>
wanted to remain cautious to
breaking OSMesa/other build<br>
configurations and it was
largely ported from previous
logic.<br>
<span><font color="#888888"><br>
Marcus<br>
</font></span>
<div>
<div>_______________________________________________<br>
Powered by <a
moz-do-not-send="true"
href="http://www.kitware.com"
target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware
open-source projects at <a
moz-do-not-send="true"
href="http://www.kitware.com/opensource/opensource.html"
target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Follow this link to
subscribe/unsubscribe:<br>
<a moz-do-not-send="true"
href="http://www.vtk.org/mailman/listinfo/vtk-developers"
target="_blank">http://www.vtk.org/mailman/listinfo/vtk-developers</a><br>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</blockquote>
<br>
</body>
</html>