[CMake] What does `cross-platform' mean?

[Jed Brown]

On Fri 2008-11-07 10:47, Bill Hoffman wrote:
> So, currently CMake assumes that the environment you are working from is  
> correctly configured.  I think adding CMakeModule would go beyond the  
> scope of what CMake should be doing.   Just like you have to have a  
> working compiler in your path, you should also have a working 
> environment.

Is my environment not `working' simply because I'm asking to link the
versions of libraries that are found at a path other than the system
defaults?  Using something else and asking the user to edit the cache is
like ignoring CC and requiring the user to edit CMAKE_C_COMPILER,
CMAKE_LINKER, CMAKE_AR, CMAKE_RANLIB, etc to support a different
toolchain.

> However, I am sure the find_* stuff can be further refined, and extra  
> error checks put in place to make sure ABI's match.  CMake works best if  
> it can find a full path to a specific library.   The idea of a command  
> that turns -L/path -lA into /path/libA.a sounds like a useful thing to  
> add.

See this:

  http://github.com/jedbrown/cmake-modules/tree/master/ResolveCompilerPaths.cmake

It will fail on pathological inputs (we need a way to preserve
shell quoting and tokenizing semantics, I don't aspire to write the
fully Posix-complient regex), but should be much better than existing
attempts (in FindPkgConfig and FindMPI).

> So, as to the title of this thread, Cross platform means Windows, OSX,  
> Linux (all distros), HPUX, IRIX, SunOS, Haiku, QNX, cygwin, mingw.

Yes, but the Find* stuff requires manual intervention (like editing
lots of advanced cache entries) when shared libraries are not available
or when the user wants to link a version in a nonstandard location (when
there is also a version in a standard location).  It's a hard problem
that apparently hasn't been addressed.  That was the root of my
question.

> As far as I can tell, the only problem you are talking about here, is  
> when you link to a static library that depends on other static libraries  
> CMake is not figuring out the depend information for you.  That  
> information has to come form somewhere.  It comes from:
>
> 1. pkg-config output, which potentially needs the above command that can  
> turn -L -l into full paths for the libraries.
>
> 2. from makefile fragments
>
> 3. If it is a cmake project from that package config stuff
>
> 4.  Other places I am sure.  There really is no standard for this.
>
> So, I would appreciate constructive feedback on what specific commands  
> and enhancements can be added to CMake in order to make the writing of  
> Find* modules easier to do and get right.  I would suggest that the  
> CMake Wiki be used to come up with some ideas.   Maybe add a section 
> here:
>
> http://www.vtk.org/Wiki/CMake#More_Topics
>
> Something like
> "Proposal for improved commands to better handle third party library  
> dependency information"

I added two pages to that section

  http://www.cmake.org/Wiki/CMake:Static_libraries
  http://www.cmake.org/Wiki/CMake:Multiple_versions

The second addresses the issue of making a module with subordinate cache
entries.

> I would also like to comment on this quote:
> "CMake while finding that many modules (most of which I don't
> actually care about) are systematically broken in niche environments
> which are disproportionately popular on clusters, an important
> demographic for my projects."
>
> CMake has been used on many cluster environments by Kitware with  
> ParaView and other software, so the issues you are having are not even  
> widespread in that environment.

Mandatory static libraries are certainly not rare.  Neither is having
multiple versions of libraries and wanting a version in a non-standard
location.  The clusters that I've used usually provide a core set of
libraries managed by a `module' tool which sets up the environment so
the correct version is found first, but if you need anything other than
the default build, you build it in your home directory and specify the
path.  On every system I've worked on, lots of users have local builds
of libraries which they want to link.

Remember that ParaView bundles it's dependencies so there is little
concern about finding the right version or tracking down dependencies.
For instance, ParaView is bundled with a forked (at least by build
system) version of HDF5.  It is not practical nor desirable for every
project to do this.

While it's acceptable for an application to bundle deps, it's certainly
not acceptable for a library where a selling point is that applications
can combine libraries.  That is, if libB and libC both depend on libD,
it's not okay for them to both bundle libD (unless they provide a
reliable way to get a non-bundled version).  AppA, on the other hand, is
guaranteed to be the end of the line so it's acceptable to bundle libB,
libC, and libD (even though AppA may never call libD directly).

> However, there is always room for improvement.  So, if you are serious
> about helping or other folks want to help with this issue, lets start
> with a wiki article and see where this goes.

Wiki started, see above.  It would be really useful for a CMake
developer to look at these modules

  http://github.com/jedbrown/cmake-modules/tree/master/ResolveCompilerPaths.cmake
  http://github.com/jedbrown/cmake-modules/tree/master/FindPackageMultipass.cmake

and decide if something similar should be added to CMake.  The most
important thing is to decide on a stable API for doing these operations
because when/if support for static libs and multiple versions is added,
almost every module will need to be updated (unless someone is very
clever).

> One last thought...   The Find* stuff in CMake is hard!   Software is  
> installed all over the place, and often times in non-standard locations.  
>  That is why there is a cache editor in CMake so if it fails to find the 
> right stuff, you can manually fix it.

Find* is definitely hard.  For static libs, I see editing the advanced
cache as a non-solution.  A nontrivial library will require the user to
fill in lots (like 40) full paths to static libraries that they may not
even know about (they're recursive dependencies).  In the best case,
this means tracking down documentation for the interface library and
learning how to link the dependency graph, then hand-resolving the link
line and writing it into the advanced cache entry.  Failing that, it
means compiling with VERBOSE=1 and, reading the linker errors, inferring
the necessary library from the names of unresolved symbols, manually
putting this library in the cache, and repeating.  Very few users have
the patience for either.

> Also, if anyone knows of a build system that does any of this, we can
> always copy what that system does.  Although, I don't think any of
> them do much of this stuff...

Yes, the build system world is kind of a mess and resolving external
libs is hard.  BuildSystem (http://petsc.cs.iit.edu/petsc/BuildSystem)
is quite project-specific, but behaves sensibly with static libs and
multiple versions, albeit often with truly horrendous link lines
obtained by concatenating multiple recursive link lines.  A lot of
hand-tuned automake scripts can deal as well.  CMake is definitely
better than these once the `build' phase is reached, but the Find* stuff
is still weak.  Hopefully that can be improved.

BTW, all the modules in my repository

  http://github.com/jedbrown/cmake-modules

are public domain and I'm willing to maintain them if they are added to
CMake.

Jed
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://www.cmake.org/pipermail/cmake/attachments/20081109/4607ba8a/attachment.pgp>