[cmake-developers] CMake alternative language

[Brad King]

On 01/14/2016 09:11 AM, Charles Huet wrote:
> Thanks for the insight !
[snip]
> but Lua seems to be a better fit for CMake now that I think about the
> constraints you listed.

Yes, Eric, thanks for the excellent post summarizing the history of this
issue and why Lua is a great candidate.

> The main point I am getting from your mail is that Kitware may not
> want to go down this route, even if provided with a working prototype,
> with the ability of slow migration by using both CMake script and Lua.

I'm not opposed to this in principle but it will be a huge change to the
ecosystem and so cannot be taken lightly.  FYI, CMake 3.0 introduced
Lua-style bracket arguments (of the form "[[...]]") specifically to
enable the possibility of embedding Lua code (or other content) inside
CMake scripts without having to do any escaping.  This could aid with
incremental transitions.

> I understand that some users are against such a change (as pointed out
> in this very thread) but I'd be willing to contribute time and effort
> towards it, if given the assurance it would not be in vain.

I think the first step is design brainstorming and discussion to see if
we can find something acceptable.

IIRC one problem with earlier attempts at introducing Lua was that CMake's
dynamic scoping is difficult to reconcile with Lua's lexical scoping.  Lua
cannot be given direct access to CMake variable scopes because Lua data
structures may outlive the CMake dynamic scopes, leading to dangling
references.

I'd also prefer an approach that allows us to offer newer versions of Lua
as they come out while still processing old scripts unchanged.  This means
that all Lua code must lie in a context where CMake knows the version of
Lua for which it was written.  I'd like to retain the possibility of
supporting multiple Lua versions to allow future such version transitions.
(Implementation symbols can be mangled to support multiple versions linked
in one program.)

I once designed the following CMake language command as a way to embed Lua
inside CMake scripts while addressing both of the above concerns.  I'm not
necessarily proposing this, but it serves as an example approach:

--------------------------------------------------------------------------
Usage:

  cmake_lua(<version> [LOCAL <local>...] CODE <code>...)

The <version> must be "5.2", the Lua language version.
Each <local> specifies a Lua local variable with the form
"var=type{value}" or "type{var}" (short for "var=type{${var}}")
where "var" is the Lua identifier for the local and "type"
determines how "value" is converted to Lua.  Valid types are:

  string: value is stored directly in a Lua string.
  list:   value is a ;-separated list of the form "a;b;c"
          to be stored in a Lua table of the form {'a','b','c'}.
  number: value is interpreted as a decimal representation of
          a floating point value to be stored in a Lua number.

All <code> arguments are concatenated into a Lua chunk with form
"<prologue> <code>... <epilogue>" where <prologue> and <epilogue>
are generated by CMake to provide <local>s by unspecified means.
The <code>... must return a single Lua table containing zero or
more identifier keys to be set as CMake variables.  Values must be
convertible by the reverse of the above type mapping.

Example:

set(deg 90)
cmake_lua(5.2 LOCAL number{deg} CODE [[
  return { sin = math.sin(deg*math.pi) }]])
message("sin(${deg}') = ${sin}')

--------------------------------------------------------------------------

Of course the above all assumes that Lua code may run simultaneously
with CMake code during configuration.  Other approaches that involve
pure-Lua programs do not have the same scoping challenges (but would
still have to deal with Lua versioning).

>> But it sounds like Brad is more interesting in a purely declarative
>> replacement for CMake.
>> That would be nice for all the editor-assist reasons. Though I’m also
>> skeptical of pulling that off (though I hope it can be done in a
>> useful way). There are other systems like that, yet, I’m using CMake
>> for a reason.

I'm raising the declarative approach here because it is an important
consideration.  See the "CMake daemon for user tools" thread for
discussion of how a declarative spec would greatly improve integration
with IDEs and other tools:

 https://cmake.org/pipermail/cmake-developers/2016-January/027413.html

A declarative spec allows tools to load and work with it easily.
A JSON document with a schema is easy to load, edit, validate, and
save, for example.  A procedural specification must be executed to be
interpreted, and only humans can really edit it.

========================================================================

Currently CMake language code is a program that runs to generate a
build specification stored within CMake's implementation that is then
used by the generators.  Since the CMake language files (CMakeLists.txt)
are procedural, projects can do all kinds of things like system
introspection, configuring files, etc. before and after calling the
commands like add_library, add_executable, and target_link_libraries
that actually specify the build rules.  The language was originally
meant only for the latter part.  In fact originally the commands were
more declarative and did not have a defined order of execution.  Then
it grew into what it is now to support the other parts.

Both the earlier attempt to use Lua that Eric mentioned and the Python
approach proposed at the beginning of this thread follow the same
approach just with another language.  I think if we are going to go
through the effort to change the status quo then we need to consider
other weaknesses of the approach than just the language.  These include
the lack of a declarative (i.e. toolable) spec and the serial nature
of the programs limiting possibility for parallel or partial evaluation.

-Brad