<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    To close the loop on this I made a go at the superbuild.
    unortunately it dies in the python build of the tools pass. This
    kind of debugging effort that I'd like to avoid! especially as
    compiling python is a simple matter and NERSC already has static
    builds for all of the other deps. My experience with VisIt, and it's
    "superbuild", has enforced the idea that such things are generally a
    bad idea and make it it much harder to deploy the software. A common
    issue with VisIt's build is that it works for developers on
    "blessed"  systems but in the wild it fails miserably. Debugging the
    superbuild requires more work than making the installs directly! In
    a way the superbuild becomes a replacement for documentation. I
    think for 4.1 release I'm going to stay with what we have: stock PV
    build, patched so that X11 deps are removed. However, I appreciate
    the info about cross compiling and superbuild process!<br>
    <br>
    ...<br>
    CMake Warning:<br>
      Manually-specified variables were not used by the project:<br>
    <br>
        CROSS_BUILD_SITE<br>
        ENABLE_cgns<br>
        ENABLE_ffmpeg<br>
        ENABLE_hdf5<br>
        ENABLE_matplotlib<br>
        ENABLE_silo<br>
        ENABLE_szip<br>
        ENABLE_visitbridge<br>
        qt_DISABLE_WEBKIT<br>
    ...<br>
    configure: creating ./config.status<br>
    config.status: creating Makefile.pre<br>
    config.status: creating Modules/Setup.config<br>
    config.status: creating Misc/python.pc<br>
    config.status: creating Modules/ld_so_aix<br>
    config.status: creating pyconfig.h<br>
    configure: WARNING: unrecognized options: --enable-static<br>
    creating Modules/Setup<br>
    creating Modules/Setup.local<br>
    creating Makefile<br>
    Current working directory cannot be established.<br>
    make[2]: *** [python/src/python-stamp/python-build] Error 1<br>
    make[1]: *** [CMakeFiles/python.dir/all] Error 2<br>
    make: *** [all] Error 2<br>
    <br>
    <br>
    <div class="moz-cite-prefix">On 03/17/2014 02:06 PM, Burlen Loring
      wrote:<br>
    </div>
    <blockquote cite="mid:532763DD.4090802@gmail.com" type="cite">
      <meta content="text/html; charset=ISO-8859-1"
        http-equiv="Content-Type">
      nice! that settles it, I should definitely be using the
      superbuild!<br>
      <br>
      My feeling is that it's probably still a good idea to look into
      the OSMesa/libGL/X11 link issues and clean it up. But hopefully
      superbuild will do the trick for our installs.<br>
      <br>
      Thanks<br>
      Burlen<br>
      <br>
      <div class="moz-cite-prefix">On 03/17/2014 01:53 PM, David E
        DeMarle wrote:<br>
      </div>
      <blockquote
cite="mid:CANjZAi8Fbf12MxpmvL=N9wOACf7DyhRF9VyqEe6cOEvAX+frkw@mail.gmail.com"
        type="cite">
        <div dir="ltr">Yes it is a real static build:
          <div>On titan@ornl:<br>
            <div>
              <div>home/demarled> ldd
                /sw/xk6/paraview/4.1.0/bin/pvbatch <br>
              </div>
              <div>        not a dynamic executable</div>
              <div>home/demarled> ldd
                /sw/xk6/paraview/4.1.0/bin/pvserver </div>
              <div>        not a dynamic executable</div>
            </div>
            <div><br>
            </div>
            <div>On mira@anl:</div>
            <div>
              <div>[demarle@miralac1 ~]$ ldd
                /soft/visualization/paraview/v4.1.0/bin/pvserver </div>
              <div>        not a dynamic executable</div>
              <div>[demarle@miralac1 ~]$ ldd
                /soft/visualization/paraview/v4.1.0/bin/pvbatch </div>
              <div>        not a dynamic executable</div>
            </div>
            <div><br>
            </div>
          </div>
          <div>I agree there are drawbacks. First and foremost is that
            XCSB is using our own python and mesa.</div>
          <div><br>
          </div>
          <div><br>
          </div>
        </div>
        <div class="gmail_extra"><br clear="all">
          <div>David E DeMarle<br>
            Kitware, Inc.<br>
            R&D Engineer<br>
            21 Corporate Drive<br>
            Clifton Park, NY 12065-8662<br>
            Phone: 518-881-4909</div>
          <br>
          <br>
          <div class="gmail_quote">On Mon, Mar 17, 2014 at 4:43 PM,
            Burlen Loring <span dir="ltr"><<a moz-do-not-send="true"
                href="mailto:burlen.loring@gmail.com" target="_blank">burlen.loring@gmail.com</a>></span>
            wrote:<br>
            <blockquote class="gmail_quote" style="margin:0 0 0
              .8ex;border-left:1px #ccc solid;padding-left:1ex">
              <div text="#000000" bgcolor="#FFFFFF"> Hi Andy, <br>
                <br>
                do you actually get static executables out of the
                superbuild (ldd pvserver reports not a dyanmic exec)?.
                If so then I really should look into it.  The reason I
                haven't is that the superbuild adds complications and
                potential failure points, which due to NERSC's
                aggressive system software upgrades, I predict I will be
                the one debugging. same with cross compiling. I feel
                that neither superbuild nor cross compile should be
                necessary on these systems, which have same cpu arch on
                login and compute nodes. On the other hand it would be
                worth the hassle if it is indeed producing static execs.<span
                  class="HOEnZb"><font color="#888888"><br>
                    <br>
                    Burlen</font></span>
                <div>
                  <div class="h5"><br>
                    <br>
                    <div>On 03/17/2014 12:12 PM, Andy Bauer wrote:<br>
                    </div>
                    <blockquote type="cite">
                      <div dir="ltr">
                        <div>
                          <div>Hey Burlen,<br>
                            <br>
                          </div>
                          The superbuild (<a moz-do-not-send="true"
                            href="http://paraview.org/Wiki/ParaView/Superbuild"
                            target="_blank">http://paraview.org/Wiki/ParaView/Superbuild</a>)
                          now does a pretty good job of building
                          statically on the Crays. Have you tried that
                          lately? If you look at what I did for
                          Garnet@ERDC's HPC center you can see how I
                          modified paths to compilers and libraries from
                          the default settings. I have a script below
                          that has the settings for doing the static
                          cross build on Garnet, in cast that helps.<br>
                          <br>
                        </div>
                        That being said, I'd still like the change of
                        not linking with X11 if using OSMesa as that
                        will also be useful to many people. <br>
                        <div><br>
                        </div>
                        <div>Regards,<br>
                          Andy<br>
                        </div>
                        <div><br>
                          ================================<br>
                          #!/bin/bash<br>
                          <br>
ParaViewSuperBuildSource=/usr/local/usp/DAAC/paraview/source/ParaView-4.1.0-All/ParaViewSuperbuild<br>
                          cd
                          /usr/local/usp/DAAC/paraview/source/ParaView-4.1.0-All<br>
                          <br>
                          mkdir tools<br>
                          mkdir cross<br>
                          <br>
                          cd tools<br>
                          #switch compiler to compile code for front end<br>
                          #module load cmake<br>
                          module unload PrgEnv-pgi<br>
                          module unload PrgEnv-gnu<br>
                          #module load git<br>
                          module load gcc<br>
                          # CXX was set to CC which isn't defined unless
                          a PrgEnv-gnu/pgi/etc module is loaded<br>
                          export CXX=/opt/gcc/4.8.0/bin/g++<br>
                          export CC=/opt/gcc/4.8.0/bin/gcc<br>
                          <br>
                          # 2/13/2014 -- took out 
                          -DParaView_URL_MD5:STRING=6882d488ac8a4622d8a9c632b1a57b44
                          \<br>
                          <br>
                          #configure settings for to build compile tools
                          only<br>
                          cmake \<br>
                            -DCROSS_BUILD_STAGE:STRING=TOOLS
                          -Dcross_target:STRING=xk7_gnu \<br>
                            -DCROSS_BUILD_SITE:STRING=garnet \<br>
                           
                          -DCMAKE_CXX_COMPILER:FILEPATH=/opt/gcc/4.8.0/bin/g++
                          \<br>
                           
                          -DCMAKE_C_COMPILER:FILEPATH=/opt/gcc/4.8.0/bin/gcc
                          \<br>
                            -DCMAKE_BUILD_TYPE:STRING=Release \<br>
                            -DBUILD_TESTING:BOOL=FALSE \<br>
                            -DENABLE_paraview:BOOL=TRUE \<br>
                            -DENABLE_boost:BOOL=TRUE \<br>
                            -DENABLE_python:BOOL=TRUE \<br>
                            -DENABLE_portfwd:BOOL=FALSE \<br>
                            -DParaView_FROM_GIT:BOOL=OFF \<br>
                            -DParaView_URL:STRING="<a
                            moz-do-not-send="true"
                            href="http://www.paraview.org/files/v4.1/ParaView-v4.1.0-source.tar.gz"
                            target="_blank">http://www.paraview.org/files/v4.1/ParaView-v4.1.0-source.tar.gz</a>"
                          \<br>
                            -DENABLE_visitbridge:BOOL=ON \<br>
                            -DENABLE_cgns=ON \<br>
                            -DENABLE_ffmpeg=ON \<br>
                            -DENABLE_hdf5=ON \<br>
                            -DENABLE_matplotlib=ON \<br>
                            -DENABLE_silo=ON \<br>
                            -DENABLE_szip=ON \<br>
                            -Dqt_DISABLE_WEBKIT=ON \<br>
                            ${ParaViewSuperBuildSource}<br>
                          <br>
                          make -j2<br>
                          <br>
                          #switch compiler to compile code for back end<br>
                          # cmake and git are already loaded when I log
                          in<br>
                          cd ../cross<br>
                          #module load cmake<br>
                          module unload gcc<br>
                          module load PrgEnv-gnu<br>
                          module load gcc/4.8.0<br>
                          #not sure why module load wasn't sufficient,
                          but ended up needing to force<br>
                          #cmake to choose the right compiler<br>
                          #export CC=/opt/cray/xt-asyncpe/5.17/bin/cc<br>
                          #export CXX=/opt/cray/xt-asyncpe/5.17/bin/CC<br>
                          #export CXX=/opt/gcc/4.8.0/bin/g++<br>
                          #export CC=/opt/gcc/4.8.0/bin/gcc<br>
                          export CXX=/opt/cray/xt-asyncpe/5.21/bin/CC<br>
                          export CC=/opt/cray/xt-asyncpe/5.21/bin/cc<br>
                          #configure settings to cross compile python,
                          (mesa - implied), and paraview<br>
                          cmake \<br>
                            -DCROSS_BUILD_STAGE:STRING=CROSS
                          -Dcross_target:STRING=xk7_gnu \<br>
                            -DCROSS_BUILD_SITE:STRING=garnet \<br>
                            -DCMAKE_BUILD_TYPE:STRING=Release \<br>
                            -DBUILD_TESTING:BOOL=FALSE \<br>
                            -DENABLE_paraview:BOOL=TRUE \<br>
                            -DENABLE_python:BOOL=TRUE \<br>
                            -DParaView_FROM_GIT:BOOL=OFF \<br>
                           
                          -DCMAKE_INSTALL_PREFIX:PATH=/usr/local/usp/DAAC/paraview/4.1.0_osmesa
                          \<br>
                            ${ParaViewSuperBuildSource}<br>
                          <br>
                          make -j2 install<br>
                          <br>
                          # with CMAKE_INSTALL_PREFIX set, I shouldn't
                          have to do the manual install...<br>
                          # next go into
                          cross/paraview/src/paraview-build and set the
                          install directory with CMake<br>
                          # then do make install, copy site-packages
                          directory from lib/paraview-4.0 into lib.<br>
                          # finally pvbatch relies on where python was
                          installed (i.e. cross/install/lib) but<br>
                          # I can copy that to someplace else and set
                          PYTHONHOME to take care of it.<br>
                          # for example:<br>
                          # cp -r cross/install/lib/python2.7
                          <install>/lib<br>
                          # export PYTHONHOME=<install><br>
                          # such that $PYTHONHOME/lib/python2.7/ has all
                          of the default .py files<br>
                          ==================<br>
                        </div>
                      </div>
                      <div class="gmail_extra"><br>
                        <br>
                        <div class="gmail_quote">On Mon, Mar 17, 2014 at
                          3:01 PM, burlen <span dir="ltr"><<a
                              moz-do-not-send="true"
                              href="mailto:burlen.loring@gmail.com"
                              target="_blank">burlen.loring@gmail.com</a>></span>
                          wrote:<br>
                          <blockquote class="gmail_quote"
                            style="margin:0 0 0 .8ex;border-left:1px
                            #ccc solid;padding-left:1ex">
                            <div text="#000000" bgcolor="#FFFFFF">
                              <div>Thanks Guys! I appreciate your eyes
                                on it.<br>
                                <br>
                                As you point out clearing
                                OPENGL_gl_LIBRARY should prevent libGL
                                dependency and link order issues. In my
                                builds I have got into the habit of
                                setting it to libOSMesa which does the
                                same. In any case this doesn't prevent
                                the X11 libs from being linked. in
                                FindOpenGL.cmake that ships with cmake,
                                X11 libs are always tacked onto
                                OPENGL_LIBRARIES, which is used in
                                Rendering/OpenGL/CMakeLists. <br>
                                <br>
                                my main motivation in digging into these
                                issues is to address a PV performance
                                issue on NERSC's Crays. Users found that
                                PV start-ups were sometimes taking a
                                very long time (eg 30 min for 48 way
                                parallel run). We tracked the issue to
                                the file system and PV's use of shared
                                libraries. With PV there are many shared
                                library dependencies (~200), and when
                                parallel pvserver's startup they all hit
                                the file system at the same time all
                                opening these shared library files. When
                                I make static executables startup time
                                is reduced reliably to a few seconds. I
                                admit static executables are an extreme
                                measure, and I hit a handful of cmake/pv
                                specific cmake related issues doing it
                                that won't be easy to address. However,
                                it's probably OK performance wise to
                                settle for "as static as possible", and
                                getting the list of shared library
                                dependencies from ~200 to ~10 should
                                make for a big improvement. Removing
                                those X11 deps when using OSMesa gets us
                                "as static as possible" and without too
                                much effort. <br>
                                <span><font color="#888888"> <br>
                                    Burlen</font></span>
                                <div>
                                  <div><br>
                                    <br>
                                    On 03/17/2014 08:03 AM, David E
                                    DeMarle wrote:<br>
                                  </div>
                                </div>
                              </div>
                              <div>
                                <div>
                                  <blockquote type="cite">
                                    <div dir="ltr">I'll take a close
                                      look at this too.
                                      <div><br>
                                      </div>
                                      <div>Like Andy said, we do get by
                                        for example in ParaView's cross
                                        compiling super build, but I'm
                                        all in for making it easier in
                                        practice to get VTK and ParaView
                                        up and running in HPC contexts.</div>
                                      <div><br>
                                      </div>
                                      <div>Thanks as always Burlen!
                                        <div><br>
                                        </div>
                                      </div>
                                    </div>
                                    <div class="gmail_extra"><br
                                        clear="all">
                                      <div>David E DeMarle<br>
                                        Kitware, Inc.<br>
                                        R&D Engineer<br>
                                        21 Corporate Drive<br>
                                        Clifton Park, NY 12065-8662<br>
                                        Phone: <a
                                          moz-do-not-send="true"
                                          href="tel:518-881-4909"
                                          value="+15188814909"
                                          target="_blank">518-881-4909</a></div>
                                      <br>
                                      <br>
                                      <div class="gmail_quote">On Mon,
                                        Mar 17, 2014 at 10:06 AM, Marcus
                                        D. Hanwell <span dir="ltr"><<a
                                            moz-do-not-send="true"
                                            href="mailto:marcus.hanwell@kitware.com"
                                            target="_blank">marcus.hanwell@kitware.com</a>></span>
                                        wrote:<br>
                                        <blockquote class="gmail_quote"
                                          style="margin:0 0 0
                                          .8ex;border-left:1px #ccc
                                          solid;padding-left:1ex">
                                          <div>On Fri, Mar 14, 2014 at
                                            5:31 PM, Burlen Loring <<a
                                              moz-do-not-send="true"
                                              href="mailto:burlen.loring@gmail.com"
                                              target="_blank">burlen.loring@gmail.com</a>>



                                            wrote:<br>
                                          </div>
                                          <div>> Hi All,<br>
                                            ><br>
                                            > While installing the
                                            past two PV releases I've
                                            noticed that VTK is linking<br>
                                            > in X11 and libGL when
                                            using OSMesa, which is
                                            acheived by VTK_USE_X=OFF
                                            and<br>
                                            >
                                            VTK_OPENGL_HAS_OSMESA=ON.
                                            This is one a of a number of
                                            issues that is making<br>
                                            > statically linked PV
                                            executables on the Cray
                                            painful. I think that the<br>
                                            > history of this is that
                                            this was useful in the past
                                            when mangled Mesa was<br>
                                            > used but currently this
                                            is the wrong thing to do
                                            because the extension<br>
                                            > manager can only use
                                            one or the other as
                                            GetProcAddress function is
                                            selected<br>
                                            > at compile time, and at
                                            run time it's used
                                            indiscriminatly regardless
                                            of<br>
                                            > actual context
                                            type(OSMesa context or other
                                            GL context) so there's
                                            currently<br>
                                            > no chance that you
                                            could safely use both. Also,
                                            linking both libraries can<br>
                                            > create a link time race
                                            condition between libGL and
                                            libOSMesa that depends<br>
                                            > on link order(eg both
                                            are .so's or .a's). I think
                                            that in a single build VTK<br>
                                            > needs to carefully use
                                            either libGL or OSMesa, but
                                            not both, and that<br>
                                            > linking against X11
                                            should be avoided when
                                            VTK_USE_X11=OFF.  I've
                                            tracked<br>
                                            > the X11 library
                                            inclusion down, it's coming
                                            from FindOpenGL.cmake, which<br>
                                            > appends X11 libraries
                                            to OPENGL_LIBRARIES
                                            indiscriminately when X11 is<br>
                                            > found. I've pushed a
                                            patch (which needs a careful
                                            review and some more<br>
                                            > testing) onto gerrit
                                            that explicitly separates
                                            OpenGL and OSMesa use in<br>
                                            > VTK(<a
                                              moz-do-not-send="true"
                                              href="http://review.source.kitware.com/#/c/14730/"
                                              target="_blank">http://review.source.kitware.com/#/c/14730/</a>).



                                            I'd like to know if others<br>
                                            > agree that this is a
                                            reasonable change? Thoughts
                                            and/or comments?<br>
                                            ><br>
                                          </div>
                                          I started looking at this
                                          briefly before I left, and hit
                                          the same<br>
                                          thing Andy mentioned (having
                                          to clear OPENGL_gl_LIBRARY to
                                          get a<br>
                                          working OSMesa). It would be
                                          great to improve this part of
                                          the build<br>
                                          system (I wasn't aware of all
                                          the history, but assumed it
                                          had<br>
                                          something to do with mangled
                                          mesa).<br>
                                          <br>
                                          I will try to find some time
                                          to take a closer look at the
                                          proposed<br>
                                          patch in the next few days,
                                          and always wanted to loop back
                                          to this<br>
                                          logic since modularization as
                                          it appeared overly complex but
                                          we also<br>
                                          wanted to remain cautious to
                                          breaking OSMesa/other build<br>
                                          configurations and it was
                                          largely ported from previous
                                          logic.<br>
                                          <span><font color="#888888"><br>
                                              Marcus<br>
                                            </font></span>
                                          <div>
                                            <div>_______________________________________________<br>
                                              Powered by <a
                                                moz-do-not-send="true"
                                                href="http://www.kitware.com"
                                                target="_blank">www.kitware.com</a><br>
                                              <br>
                                              Visit other Kitware
                                              open-source projects at <a
                                                moz-do-not-send="true"
                                                href="http://www.kitware.com/opensource/opensource.html"
                                                target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
                                              <br>
                                              Follow this link to
                                              subscribe/unsubscribe:<br>
                                              <a moz-do-not-send="true"
href="http://www.vtk.org/mailman/listinfo/vtk-developers"
                                                target="_blank">http://www.vtk.org/mailman/listinfo/vtk-developers</a><br>
                                              <br>
                                            </div>
                                          </div>
                                        </blockquote>
                                      </div>
                                      <br>
                                    </div>
                                  </blockquote>
                                  <br>
                                </div>
                              </div>
                            </div>
                          </blockquote>
                        </div>
                        <br>
                      </div>
                    </blockquote>
                    <br>
                  </div>
                </div>
              </div>
            </blockquote>
          </div>
          <br>
        </div>
      </blockquote>
      <br>
    </blockquote>
    <br>
  </body>
</html>