[vtk-developers] X11 and libGL always linked

Burlen Loring burlen.loring at gmail.com
Tue Mar 18 15:52:55 EDT 2014


To close the loop on this I made a go at the superbuild. unortunately it 
dies in the python build of the tools pass. This kind of debugging 
effort that I'd like to avoid! especially as compiling python is a 
simple matter and NERSC already has static builds for all of the other 
deps. My experience with VisIt, and it's "superbuild", has enforced the 
idea that such things are generally a bad idea and make it it much 
harder to deploy the software. A common issue with VisIt's build is that 
it works for developers on "blessed"  systems but in the wild it fails 
miserably. Debugging the superbuild requires more work than making the 
installs directly! In a way the superbuild becomes a replacement for 
documentation. I think for 4.1 release I'm going to stay with what we 
have: stock PV build, patched so that X11 deps are removed. However, I 
appreciate the info about cross compiling and superbuild process!

...
CMake Warning:
   Manually-specified variables were not used by the project:

     CROSS_BUILD_SITE
     ENABLE_cgns
     ENABLE_ffmpeg
     ENABLE_hdf5
     ENABLE_matplotlib
     ENABLE_silo
     ENABLE_szip
     ENABLE_visitbridge
     qt_DISABLE_WEBKIT
...
configure: creating ./config.status
config.status: creating Makefile.pre
config.status: creating Modules/Setup.config
config.status: creating Misc/python.pc
config.status: creating Modules/ld_so_aix
config.status: creating pyconfig.h
configure: WARNING: unrecognized options: --enable-static
creating Modules/Setup
creating Modules/Setup.local
creating Makefile
Current working directory cannot be established.
make[2]: *** [python/src/python-stamp/python-build] Error 1
make[1]: *** [CMakeFiles/python.dir/all] Error 2
make: *** [all] Error 2


On 03/17/2014 02:06 PM, Burlen Loring wrote:
> nice! that settles it, I should definitely be using the superbuild!
>
> My feeling is that it's probably still a good idea to look into the 
> OSMesa/libGL/X11 link issues and clean it up. But hopefully superbuild 
> will do the trick for our installs.
>
> Thanks
> Burlen
>
> On 03/17/2014 01:53 PM, David E DeMarle wrote:
>> Yes it is a real static build:
>> On titan at ornl:
>> home/demarled> ldd /sw/xk6/paraview/4.1.0/bin/pvbatch
>>         not a dynamic executable
>> home/demarled> ldd /sw/xk6/paraview/4.1.0/bin/pvserver
>>         not a dynamic executable
>>
>> On mira at anl:
>> [demarle at miralac1 ~]$ ldd 
>> /soft/visualization/paraview/v4.1.0/bin/pvserver
>>         not a dynamic executable
>> [demarle at miralac1 ~]$ ldd 
>> /soft/visualization/paraview/v4.1.0/bin/pvbatch
>>         not a dynamic executable
>>
>> I agree there are drawbacks. First and foremost is that XCSB is using 
>> our own python and mesa.
>>
>>
>>
>> David E DeMarle
>> Kitware, Inc.
>> R&D Engineer
>> 21 Corporate Drive
>> Clifton Park, NY 12065-8662
>> Phone: 518-881-4909
>>
>>
>> On Mon, Mar 17, 2014 at 4:43 PM, Burlen Loring 
>> <burlen.loring at gmail.com <mailto:burlen.loring at gmail.com>> wrote:
>>
>>     Hi Andy,
>>
>>     do you actually get static executables out of the superbuild (ldd
>>     pvserver reports not a dyanmic exec)?. If so then I really should
>>     look into it.  The reason I haven't is that the superbuild adds
>>     complications and potential failure points, which due to NERSC's
>>     aggressive system software upgrades, I predict I will be the one
>>     debugging. same with cross compiling. I feel that neither
>>     superbuild nor cross compile should be necessary on these
>>     systems, which have same cpu arch on login and compute nodes. On
>>     the other hand it would be worth the hassle if it is indeed
>>     producing static execs.
>>
>>     Burlen
>>
>>
>>     On 03/17/2014 12:12 PM, Andy Bauer wrote:
>>>     Hey Burlen,
>>>
>>>     The superbuild (http://paraview.org/Wiki/ParaView/Superbuild)
>>>     now does a pretty good job of building statically on the Crays.
>>>     Have you tried that lately? If you look at what I did for
>>>     Garnet at ERDC's HPC center you can see how I modified paths to
>>>     compilers and libraries from the default settings. I have a
>>>     script below that has the settings for doing the static cross
>>>     build on Garnet, in cast that helps.
>>>
>>>     That being said, I'd still like the change of not linking with
>>>     X11 if using OSMesa as that will also be useful to many people.
>>>
>>>     Regards,
>>>     Andy
>>>
>>>     ================================
>>>     #!/bin/bash
>>>
>>>     ParaViewSuperBuildSource=/usr/local/usp/DAAC/paraview/source/ParaView-4.1.0-All/ParaViewSuperbuild
>>>     cd /usr/local/usp/DAAC/paraview/source/ParaView-4.1.0-All
>>>
>>>     mkdir tools
>>>     mkdir cross
>>>
>>>     cd tools
>>>     #switch compiler to compile code for front end
>>>     #module load cmake
>>>     module unload PrgEnv-pgi
>>>     module unload PrgEnv-gnu
>>>     #module load git
>>>     module load gcc
>>>     # CXX was set to CC which isn't defined unless a
>>>     PrgEnv-gnu/pgi/etc module is loaded
>>>     export CXX=/opt/gcc/4.8.0/bin/g++
>>>     export CC=/opt/gcc/4.8.0/bin/gcc
>>>
>>>     # 2/13/2014 -- took out
>>>     -DParaView_URL_MD5:STRING=6882d488ac8a4622d8a9c632b1a57b44 \
>>>
>>>     #configure settings for to build compile tools only
>>>     cmake \
>>>       -DCROSS_BUILD_STAGE:STRING=TOOLS -Dcross_target:STRING=xk7_gnu \
>>>       -DCROSS_BUILD_SITE:STRING=garnet \
>>>     -DCMAKE_CXX_COMPILER:FILEPATH=/opt/gcc/4.8.0/bin/g++ \
>>>     -DCMAKE_C_COMPILER:FILEPATH=/opt/gcc/4.8.0/bin/gcc \
>>>       -DCMAKE_BUILD_TYPE:STRING=Release \
>>>       -DBUILD_TESTING:BOOL=FALSE \
>>>       -DENABLE_paraview:BOOL=TRUE \
>>>       -DENABLE_boost:BOOL=TRUE \
>>>       -DENABLE_python:BOOL=TRUE \
>>>       -DENABLE_portfwd:BOOL=FALSE \
>>>       -DParaView_FROM_GIT:BOOL=OFF \
>>>      
>>>     -DParaView_URL:STRING="http://www.paraview.org/files/v4.1/ParaView-v4.1.0-source.tar.gz"
>>>     \
>>>       -DENABLE_visitbridge:BOOL=ON \
>>>       -DENABLE_cgns=ON \
>>>       -DENABLE_ffmpeg=ON \
>>>       -DENABLE_hdf5=ON \
>>>       -DENABLE_matplotlib=ON \
>>>       -DENABLE_silo=ON \
>>>       -DENABLE_szip=ON \
>>>       -Dqt_DISABLE_WEBKIT=ON \
>>>       ${ParaViewSuperBuildSource}
>>>
>>>     make -j2
>>>
>>>     #switch compiler to compile code for back end
>>>     # cmake and git are already loaded when I log in
>>>     cd ../cross
>>>     #module load cmake
>>>     module unload gcc
>>>     module load PrgEnv-gnu
>>>     module load gcc/4.8.0
>>>     #not sure why module load wasn't sufficient, but ended up
>>>     needing to force
>>>     #cmake to choose the right compiler
>>>     #export CC=/opt/cray/xt-asyncpe/5.17/bin/cc
>>>     #export CXX=/opt/cray/xt-asyncpe/5.17/bin/CC
>>>     #export CXX=/opt/gcc/4.8.0/bin/g++
>>>     #export CC=/opt/gcc/4.8.0/bin/gcc
>>>     export CXX=/opt/cray/xt-asyncpe/5.21/bin/CC
>>>     export CC=/opt/cray/xt-asyncpe/5.21/bin/cc
>>>     #configure settings to cross compile python, (mesa - implied),
>>>     and paraview
>>>     cmake \
>>>       -DCROSS_BUILD_STAGE:STRING=CROSS -Dcross_target:STRING=xk7_gnu \
>>>       -DCROSS_BUILD_SITE:STRING=garnet \
>>>       -DCMAKE_BUILD_TYPE:STRING=Release \
>>>       -DBUILD_TESTING:BOOL=FALSE \
>>>       -DENABLE_paraview:BOOL=TRUE \
>>>       -DENABLE_python:BOOL=TRUE \
>>>       -DParaView_FROM_GIT:BOOL=OFF \
>>>     -DCMAKE_INSTALL_PREFIX:PATH=/usr/local/usp/DAAC/paraview/4.1.0_osmesa
>>>     \
>>>       ${ParaViewSuperBuildSource}
>>>
>>>     make -j2 install
>>>
>>>     # with CMAKE_INSTALL_PREFIX set, I shouldn't have to do the
>>>     manual install...
>>>     # next go into cross/paraview/src/paraview-build and set the
>>>     install directory with CMake
>>>     # then do make install, copy site-packages directory from
>>>     lib/paraview-4.0 into lib.
>>>     # finally pvbatch relies on where python was installed (i.e.
>>>     cross/install/lib) but
>>>     # I can copy that to someplace else and set PYTHONHOME to take
>>>     care of it.
>>>     # for example:
>>>     # cp -r cross/install/lib/python2.7 <install>/lib
>>>     # export PYTHONHOME=<install>
>>>     # such that $PYTHONHOME/lib/python2.7/ has all of the default
>>>     .py files
>>>     ==================
>>>
>>>
>>>     On Mon, Mar 17, 2014 at 3:01 PM, burlen <burlen.loring at gmail.com
>>>     <mailto:burlen.loring at gmail.com>> wrote:
>>>
>>>         Thanks Guys! I appreciate your eyes on it.
>>>
>>>         As you point out clearing OPENGL_gl_LIBRARY should prevent
>>>         libGL dependency and link order issues. In my builds I have
>>>         got into the habit of setting it to libOSMesa which does the
>>>         same. In any case this doesn't prevent the X11 libs from
>>>         being linked. in FindOpenGL.cmake that ships with cmake, X11
>>>         libs are always tacked onto OPENGL_LIBRARIES, which is used
>>>         in Rendering/OpenGL/CMakeLists.
>>>
>>>         my main motivation in digging into these issues is to
>>>         address a PV performance issue on NERSC's Crays. Users found
>>>         that PV start-ups were sometimes taking a very long time (eg
>>>         30 min for 48 way parallel run). We tracked the issue to the
>>>         file system and PV's use of shared libraries. With PV there
>>>         are many shared library dependencies (~200), and when
>>>         parallel pvserver's startup they all hit the file system at
>>>         the same time all opening these shared library files. When I
>>>         make static executables startup time is reduced reliably to
>>>         a few seconds. I admit static executables are an extreme
>>>         measure, and I hit a handful of cmake/pv specific cmake
>>>         related issues doing it that won't be easy to address.
>>>         However, it's probably OK performance wise to settle for "as
>>>         static as possible", and getting the list of shared library
>>>         dependencies from ~200 to ~10 should make for a big
>>>         improvement. Removing those X11 deps when using OSMesa gets
>>>         us "as static as possible" and without too much effort.
>>>
>>>         Burlen
>>>
>>>
>>>         On 03/17/2014 08:03 AM, David E DeMarle wrote:
>>>>         I'll take a close look at this too.
>>>>
>>>>         Like Andy said, we do get by for example in ParaView's
>>>>         cross compiling super build, but I'm all in for making it
>>>>         easier in practice to get VTK and ParaView up and running
>>>>         in HPC contexts.
>>>>
>>>>         Thanks as always Burlen!
>>>>
>>>>
>>>>         David E DeMarle
>>>>         Kitware, Inc.
>>>>         R&D Engineer
>>>>         21 Corporate Drive
>>>>         Clifton Park, NY 12065-8662
>>>>         Phone: 518-881-4909 <tel:518-881-4909>
>>>>
>>>>
>>>>         On Mon, Mar 17, 2014 at 10:06 AM, Marcus D. Hanwell
>>>>         <marcus.hanwell at kitware.com
>>>>         <mailto:marcus.hanwell at kitware.com>> wrote:
>>>>
>>>>             On Fri, Mar 14, 2014 at 5:31 PM, Burlen Loring
>>>>             <burlen.loring at gmail.com
>>>>             <mailto:burlen.loring at gmail.com>> wrote:
>>>>             > Hi All,
>>>>             >
>>>>             > While installing the past two PV releases I've
>>>>             noticed that VTK is linking
>>>>             > in X11 and libGL when using OSMesa, which is acheived
>>>>             by VTK_USE_X=OFF and
>>>>             > VTK_OPENGL_HAS_OSMESA=ON. This is one a of a number
>>>>             of issues that is making
>>>>             > statically linked PV executables on the Cray painful.
>>>>             I think that the
>>>>             > history of this is that this was useful in the past
>>>>             when mangled Mesa was
>>>>             > used but currently this is the wrong thing to do
>>>>             because the extension
>>>>             > manager can only use one or the other as
>>>>             GetProcAddress function is selected
>>>>             > at compile time, and at run time it's used
>>>>             indiscriminatly regardless of
>>>>             > actual context type(OSMesa context or other GL
>>>>             context) so there's currently
>>>>             > no chance that you could safely use both. Also,
>>>>             linking both libraries can
>>>>             > create a link time race condition between libGL and
>>>>             libOSMesa that depends
>>>>             > on link order(eg both are .so's or .a's). I think
>>>>             that in a single build VTK
>>>>             > needs to carefully use either libGL or OSMesa, but
>>>>             not both, and that
>>>>             > linking against X11 should be avoided when
>>>>             VTK_USE_X11=OFF.  I've tracked
>>>>             > the X11 library inclusion down, it's coming from
>>>>             FindOpenGL.cmake, which
>>>>             > appends X11 libraries to OPENGL_LIBRARIES
>>>>             indiscriminately when X11 is
>>>>             > found. I've pushed a patch (which needs a careful
>>>>             review and some more
>>>>             > testing) onto gerrit that explicitly separates OpenGL
>>>>             and OSMesa use in
>>>>             > VTK(http://review.source.kitware.com/#/c/14730/). I'd
>>>>             like to know if others
>>>>             > agree that this is a reasonable change? Thoughts
>>>>             and/or comments?
>>>>             >
>>>>             I started looking at this briefly before I left, and
>>>>             hit the same
>>>>             thing Andy mentioned (having to clear OPENGL_gl_LIBRARY
>>>>             to get a
>>>>             working OSMesa). It would be great to improve this part
>>>>             of the build
>>>>             system (I wasn't aware of all the history, but assumed
>>>>             it had
>>>>             something to do with mangled mesa).
>>>>
>>>>             I will try to find some time to take a closer look at
>>>>             the proposed
>>>>             patch in the next few days, and always wanted to loop
>>>>             back to this
>>>>             logic since modularization as it appeared overly
>>>>             complex but we also
>>>>             wanted to remain cautious to breaking OSMesa/other build
>>>>             configurations and it was largely ported from previous
>>>>             logic.
>>>>
>>>>             Marcus
>>>>             _______________________________________________
>>>>             Powered by www.kitware.com <http://www.kitware.com>
>>>>
>>>>             Visit other Kitware open-source projects at
>>>>             http://www.kitware.com/opensource/opensource.html
>>>>
>>>>             Follow this link to subscribe/unsubscribe:
>>>>             http://www.vtk.org/mailman/listinfo/vtk-developers
>>>>
>>>>
>>>
>>>
>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20140318/77eec9f6/attachment-0002.html>


More information about the vtk-developers mailing list