[CMake] Is CMake powerful enough?

[Oliver Kullmann]

Hello,

Thanks for the many answers!

First some remarks to specific remarks I got and then what seems
best to do.

-------------------------

Alan W. Irwin wrote:

> I always run cmake with a separate build tree so one set of
> terminal windows is open in various sub-directories of the source tree and
> one set in various sub-directories of the build tree, and I now think of
> this as the natural way to develop.

I can hardly imagine this to be practical in my case:

1. There are really quite a few directories, all nested ("recursively"),
and always mirroring the directory movements would be a pain (and
error prone).

2. And then there is the concept of an "active library", which roughly
says that the users of the library have *all the power* the developer
have --- and since the users very likely want it convenient, also
the developer needs to have it very convenient ;-)

(((Testing, version control etc., packaging is not something reserved
to the developers side, and the user then get only the "distribution",
but the user get a full clone (with version control history etc.)
of my clone, and then they can create distributions etc. exactly
as I can do --- this is not so trivially achieved (at least this
is my experience), but we are getting closer to it (for example
creating a release becomes trivial --- just clone your own clone!).
(Sorry for making here propaganda for a concept which I cannot
really demonstrate (yet); but actually the build system is a very
important part of it, and thus I feel these kind of discussions
are not completely wrong here.))))

But it seems that having a "master script" which runs "CMake" and "make"
appropriately should do the job (see below).

> Also, you mentioned testing.  For that, check out the ADD_TEST command. I
> know ctest (or the equivalent "make test") works in the top directory of the
> build-tree, and I assume it also works in subdirectories of that tree
> although I haven't actually tried that.

The whole directory structure issue still worries me, but we'll see.

Thanks Alan!

---------------------------------------------------------

Alexander Neundorf wrote:

> You can detect directories and/or files with cmake, using IF(EXISTS) or
> FILE(GLOB). This way you can collect at cmake time the directories or files
> you want to build. If you then add a new file/directory, you will have
> to "touch" any of the cmake files to force cmake to run again if you
> enter "make" (or run cmake directly).
> But I would not recommend this (because people will forget to rerun cmake or
> to touch a file).

I got the impression, that having that "master script" (which for example
is responsible for creating new files or new directories) together with
the cmake possibilities should do the job (in a convenient and safe way).

> > It seems the basic
> > assumption is that of having a build-directory (used only once), where all
> > sort of stuff can be dumped to, and then with the final "make install" the
> > "relevant" files are moved elsewhere?

> Yes.
> But why would you care about helper files in your build tree ?

Yet I know about every little file what it's good for and where it
belongs (like a good father, I care for my little files).
The triad "source -> build -> install" is like a purgatory, where the
soul (the source) must go through hell (the build) until it emerges
in purified form (the install), however here I feel more like a modern
manager, who doesn't like to have idle files sitting around where he
doesn't know what they are doing.

Irony aside, it's hard to say what are the consequences of all such
design decisions.

> But you can set up the directories where libraries and executables go
> explicitely by setting the EXECUTABLE_OUTPUT_PATH and LIBRARY_OUTPUT_PATH.

That I will try: Having as much control as possible, and the rest is
hidden away in the dungeon.

> While it takes a bit to get used to, building out-of-source has many
> advantages and once your used to it you don't want to build in-source
> anymore.

That's perfectly clear.

For example:

> -you can simply remove the buildtree and rerun cmake so everything's fresh
again

Here my current system (not having a "build tree" and an "install tree",
but a more complex system for all the different types of files) pays off:
I can easily remove for example the dependency-files for one submodule or
for a sub-sub-module, or for the whole library, or I can inspect them --- 
and actually from time to time it's needed.

But perhaps in the future cmake might offer a bit more control (I'm not
the only control freak out there).

Thanks Alex!

----------------------------------------------------------------

Brandon Van Every wrote:

> Oliver sounds like
> he wants "crazy" dynamic everything with all sorts of elegant CS
> interfaces and research possibilities etc.

That's correct, I try my best to become a crazy scientist (however
this is not so easy(!) --- I'm not there yet).

> Some people think that a fully featured, elegant, abstract, powerful
> programming is necessary for a build system.  They might be attracted
> to SCons, for instance, because it is based on Python.

Didn't know that. I have just looked at their documentation; looks
not unreasonable. However, since I think scripting languages should
not be treated as full-blown programming languages, I always had
(and have) problems with Python.

> Myself, I've gotten very irritated when looking at functional
> programming languages, only to see build systems based on untyped
> shells using incredibly error prone system tools.

It's also my experience that functional programming in some respect
is quite advanced, and in some others quite backward.

One could say, that in my case one should just use of infrastructure
what's easiest, but I don't believe in wasting my time with using
inferior things, and I got interested in the idea of "active libraries"
(which (for me) comes from the field of generative programming (in C++)),
so that I want to create a complete novel *system*, not just another
library.

Thanks Brandon!

--------------------------------------------------------------

Bill Hoffman wrote:

> > shells using incredibly error prone system tools.  I think the SML/NJ
> > guys got sick of it and did something about it, they made some kind of
> > build tool, but I've never checked it out.
> I am not sure I agree here.   I believe that Oliver's research is in:
> "SAT solving", and not in build system maintenance.

Actually, I'm in a kind of schizoid situation, where on the one hand
I "just want to do SAT solving", not being bothered by arcane details,
but on the other hand also the system side is interesting, and then it
seems the two side cannot be really separated. (So that I hold the
believe that with better build system we will also get better
"SAT solving".)

> I never liked build systems that just find
> files.  You create a foo.cxx and it gets built into the system.   If you
> just
> wrote 100+ lines of code, is it that hard to specify one extra line in
> a build file to let the build system know that you want to build all that
> code you just wrote?

If I just look at the day today, I progressed somewhat in 5 search projects,
had short discussions with one student, extensive discussions with two
students, did some administrative work, several articles read, e-mail,
etc. ... (this will not be different from many other people, but
I just want to make a concrete example) --- so there are endless
distractions, programming is just something done besides many other
activities (writing this e-mail, I'm also reviewing 3 articles),
and I make errors and actually forget this and that. Having then
automatised something away, really makes me more productive,
and there is also the psychological side, it gives my the 
"satisfaction (kick(?!)) of automatisation".

Even more important, abstraction/automatisation is not just getting
rid off something, but it also enters a higher level, you might
think on a higher sphere.

(And it makes me feeling bad if I think I'm doing work which I
shouldn't do because a machine could do it.)

> Since your target
> platform
> is linux, you could create a few shell aliases or scripts that allow you to
> build from the source tree even with an out of source build.  I know the
> KDE folks do that.

This is what I want to do (with that "master script").

> I would suggest that you that you try CMake on some small subset
> of your system, and see how it goes.

Because of the "holistic" character of the system, I don't see a possibility
to take out some small subset; but I will try to simulate the functionality
we have achieved so far for the main part, the generic makefile, and see 
how far I get (in this way, at least I will learn to understand the current 
system a bit better ;-)).

Thanks Bill!

----------------------------------------------------------------------

Eric Noulard wrote:

> I think, that with some amount of time (not so much) for defining
> your own WHATEVER_EXECUTE CMake MACROs
> you'll get the power of what you add before
> with shell running in Makefile.

Should be possible; definitely it looks better than macro
definition in makefiles.

> You may try my example with the attached file
> and run

> cmake -P python_execute.cmake

> This example is a toy example, I am sure you
> may accomplish something far better and useful.


> MACRO(PYTHON_EXECUTE CODE RESULT ERROR)
>    SET(TMPFILE "/tmp/python_execute.py")
>    FILE(WRITE ${TMPFILE} ${CODE})
>    EXECUTE_PROCESS(COMMAND /usr/sfw/bin/python ${TMPFILE}
> 		TIMEOUT 10
> 		RESULT_VARIABLE PYTHON_RESULT
> 		OUTPUT_VARIABLE ${RESULT}
> 		ERROR_VARIABLE  ${ERROR}
> 		OUTPUT_STRIP_TRAILING_WHITESPACE)
>    FILE(REMOVE ${TMPFILE})
> ENDMACRO(PYTHON_EXECUTE)
> PYTHON_EXECUTE("import time\nprint 'todays year is:'+ time.strftime('%Y',time.localtime())" MYVAR ERROR)
> MESSAGE("MYVAR = ${MYVAR}")
> MESSAGE("ERROR = ${ERROR}")

When using "/usr/bin/python" in the script, I get

MYVAR = todays year is:2007
ERROR =

however with the script as given, which doesn't find a python
interpreter,
> /usr/sfw/bin/python
bash: /usr/sfw/bin/python: No such file or directory

I just get 

MYVAR =
ERROR =
> echo $?
0

No error? (Apparently error output didn't work?)

I know, it's always possible to write such scripts, but
it's quite some work to make it work under many circumstances.
But I guess writing all kinds of "subroutines" should be much
easier in cmake than in make.

Thanks Eric!

----------------------------------------------------------

Gonzalo Garramuño wrote:

> CMake provides FILE( GLOB ...) and similar
> constructs to do what you want.

Should suffice for most tasks.

Regarding the question of "power", I'll have to make my own
experiences, but good to hear that it works for you.

> I am attaching a simple 'mk' bash script I use to run cmake (and normal
> makefiles) in a cross-platform way (and to simplify the creating of
> out-of-source builds) and a simple optional FindBuildDir.cmake that,
> optionally, works with it.
> I think you'll find that useful in understanding how cmake can be made
> to work for a large project.

Interesting; never seen that "usage()"-syntax?!? I see, something to learn.
I'll guess I'll definitely make use of your script!

I don't understand the role of FindBuildDir.cmake? How do you use it?
It appears to be a macro collection, plus some code which gets
executed (but when and how)?

The division of work between the "master script" and cmake needs to
be figured out, but the main idea seems to me that the master script
is just there to call the right cmake or make, and to provide it
just with the necessary parameters.

Thanks Gonzalo!

---------------------------------------------------------

Brandon Van Every wrote:


> No guarantees that it'll keep working, so thanks for reminding me to
> make a feature request in the bug tracker.

Let's hope the best!

---------------------------------------------------------

Puuh; so well, then it seems clear what to do; only somebody
needed for the job (whoops, that seems to be me).

Thanks a lot for all the input! It will now take a while,
but I guess sooner or later you will see me again on this
list.

Oliver


-- 
Dr. Oliver Kullmann
Computer Science Department
University of Wales Swansea
Faraday Building, Singleton Park
Swansea SA2 8PP, UK
http://cs-svr1.swan.ac.uk/~csoliver/