<div dir="ltr">Hi,<div dir="ltr"></div><div dir="ltr"><div dir="ltr"><div><span style="line-height:19.5px"><br></span></div><div><span style="font-size:13px;line-height:19.5px">> * The cmState infrastructure builds on a "snapshot" design with a goal of</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px"> being able to "fork" configuration/generation temporarily and then revert</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px"> back, and to be able to re-start configuration from the middle. These</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px"> goals may be incompatible with any language whose implementation we do</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px"> not fully control unless it is allowed to execute only in isolated and</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px"> independent snippets. These are not hard goals, but it is a trade-off</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px"> to keep in mind. Stephen may be able to elaborate more on the snapshot</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px"> approach if needed.</span><span style="line-height:19.5px"><br></span></div><div><span style="line-height:19.5px"><br></span></div></div></div><div dir="ltr"><div dir="ltr"><div><span style="line-height:19.5px">I think these goals aim towards a faster configure, and the ability to only partly reconfigure, right ?</span></div></div></div><div dir="ltr"><div dir="ltr"><div><span style="line-height:19.5px"><br></span></div><div><span style="font-size:13px;line-height:19.5px">> * A problem with the current design is that the entire configuration process</span><br></div><div><span style="font-size:13px;line-height:19.5px"> is logically serial making parallel evaluation hard or impossible. In</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px"> many cases each add_subdirectory can be processed independently, but this</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px"> will require semantic changes to allow.</span><span style="line-height:19.5px"><br></span></div></div></div><div dir="ltr"><div dir="ltr"></div><div><br></div><div>I know I am largely biased by the project I work on, but I do not see how parallel evaluation woud be a huge benefit. We basically have a "core" library, and lots of libs that link against it (and other libs, such as third parties). </div><div>How would a target B that depends on lib A be processed in this case ? Wouldn't the evaluation of the link_libraries of B be waiting for the evaluation of A to finish ? </div><div>If this is the case, then parallel evaluation would be only a slight benefit (again, heavily biased by the project I work on).</div><div>And how would that work with CMakeLists that affect their parent scope ?</div></div><div dir="ltr"><div><br></div><div><span style="font-size:13px;line-height:19.5px">> Ideally most of the specification (sources, libraries, executables, etc.)</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px">should be in a pure format that can be evaluated without side effects (e.g.</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px">declarative or functional).</span><br></div><div><span style="font-size:13px;line-height:19.5px"><br></span></div></div><div dir="ltr"><div><span style="line-height:19.5px">I'm not sure I understand how this could be done without losing a lot of what CMake offers, such as copying or generating files.</span></div><div><span style="line-height:19.5px">I'm leaning towards a declarative approach as it is quite easy to learn (since you declare objects, and every C++ programmer I know is familiar with those), and functional, while powerfull and very elegant for some things, is quite foreign to C+ programmers as a whole (I know C++11 adds this, but it will be years until this can be used in production for lots of C++ users, because of old compilers that are still supported for business reasons).</span></div><div>Moreover, object-oriented is easy to introspect, offering an abstract representation for IDEs and tools.</div><div><br></div><div>It seems you are leaning towards pure functional, but I do not see how this would work with the current way CMake handles variables and scope, could you elaborate ?</div></div><div dir="ltr"><div><br></div><div><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px">> I do not think we should have the build specification depend on processing</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px">code like this. It is not compatible with cmake-gui where the configuration</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px">and generation steps are triggered by the user.</span><br></div><div><span style="font-size:13px;line-height:19.5px"><br></span></div></div><div dir="ltr"><div><span style="font-size:13px;line-height:19.5px">My POC is far from complete, and what I had in mind to keep the CMake-gui approach was to make generating optional in CLI through a parameter. Thus executing the script would only configure, which would populate the CMakeCache, allowing the user to modify variables and such, then configure. <br>To clarify, only the following lines </span><span style="font-size:13px;line-height:19.5px">should be considered when looking at the POC. </span></div></div><div dir="ltr"><div><span style="font-size:13px;line-height:19.5px">> myProject=cmake.Project("MyTe</span><span style="font-size:13px;line-height:19.5px">stProject")</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px">> myProject.targets=[cmake.Shar</span><span style="font-size:13px;line-height:19.5px">edLibrary("testLibrary",["lib.</span><span style="font-size:13px;line-height:19.5px">cxx"])]</span><span style="font-size:13px;line-height:19.5px"><br></span></div><div><br></div><div><span style="font-size:13px;line-height:19.5px"><br></span></div><div><span style="font-size:13px;line-height:19.5px">> In summary, I think work in this direction should first focus on designing</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px">a declarative (or functional) specification format where most of the project</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px">information can be specified. Then a cmake-language command can be written</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px">to load and evaluate a specification file (as a transition). Finally we</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px">could look at replacing the entry-point language with something else. At</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px">that point we could have closures passed as parameters to the evaluation of</span><br style="font-size:13px;line-height:19.5px"><span style="font-size:13px;line-height:19.5px">the pure spec in order to get custom generate-time logic.</span><span style="font-size:13px;line-height:19.5px"><br></span></div><div><span style="font-size:13px;line-height:19.5px"><br></span></div></div><div dir="ltr"><div><span style="font-size:13px;line-height:19.5px">It seems you have in mind to write a new CMake language. I'd rather see an existing language used, such as OCaml or lisp if you want a pure functional language. Those are supported by SWIG so prototyping would be easy. </span></div><div><span style="font-size:13px;line-height:19.5px">Javascript is even an option that is not <i>completely</i> crazy, as it has a C-Style syntax so it would be an easy language to write in for C++ programmers.</span></div><div><span style="font-size:13px;line-height:19.5px"><br></span></div><div><span style="font-size:13px;line-height:19.5px">I think using an existing language has quite a few benefits:</span></div><div><span style="font-size:13px;line-height:19.5px">* no need to design or implement a language</span></div><div><span style="font-size:13px;line-height:19.5px">* lots of existing resources on how to do things</span></div><div><span style="font-size:13px;line-height:19.5px">* lots of existing libraries (data structures, file system operations, etc)</span></div></div><div>* tooling (auto-complete, debugging, profiling, linting, etc)</div><div><br></div><div>Maybe I should take my POC further, in order to see the failings of this approach, and maybe the same should be done for a more functional approach. </div><div>CMake's own buildsystem seems like a good testing ground for this, but it is a little too big for a first go, do you know of a small CMake-based project that would be better suited ?</div><div dir="ltr"><br></div><div>I don't have a clear view of what a pure functional CMake would look like, but if you give me some mock code, I could give a try at bringing some pure functional language up to the level of my POC and we could use it as a more cconcrete discussion support.</div><div><br></div><div>Best</div><div dir="ltr"><br><div class="gmail_quote"><div dir="ltr">Le lun. 11 janv. 2016 à 10:54, Petr Kmoch <<a href="mailto:petr.kmoch@gmail.com" target="_blank">petr.kmoch@gmail.com</a>> a écrit :<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div>Hi all.<br><br></div>I'd like to voice my opinion as a somewhat advanced CMake user here.<br><br></div>For me, one of the strongest points of CMake is the fact that its project specification is procedural rather than declarative. In my current job, for example, we have a significant framework built in CMake which handles a lot of unique setups we have (which were largely inherited from a previous inhouse buildsystem). Yes, the end result of our framework is that the CMakeLists consist mostly of declarative commands from our framework, but the implementation of these commands is heavily procedural. I am convinced that if CMake didn't give us the procedural power
required to make this work, we couldn't have adopted it. (I had
previously tried emulating bits of this setup in a more declarative
system and failed miserably).<br><br></div><div>Of course (having written much of this framework I'm talking about above), I know all too well that a better front-end language would do a world of good for CMake. I also understand that taking a more declarative approach could help that, and I'm not opposed to such a change in principle. However, please take care not to throw the baby out with the bathwater and nerf the expressiveness of what can be done with CMake (in a procedural way).<br><br></div><div>If I understand Brad's suggestion correctly, it would amount to a (possibly empty) procedural step being used to generate a declarative description of the buildsystem. This would work well in our scenario, I believe, as long as that procedural step could be sufficiently modularised on the client side.<br><br></div><div>I fully support introducing an alternative input language to CMake and taking all steps necessary for this to happen, but please do this in a way which will not restrict what CMake is capable of doing.<br><br></div></div><div dir="ltr"><div>Petr<br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jan 8, 2016 at 5:30 PM, Brad King <span dir="ltr"><<a href="mailto:brad.king@kitware.com" target="_blank">brad.king@kitware.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Charles,<br>
<br>
Thanks for your efforts in exploring this topic. CMake's current language<br>
grew incrementally out of something that was not originally intended as a<br>
programming language. The cmState refactoring Stephen Kelly has started<br>
is a huge step toward enabling alternative languages, but there is a long<br>
way to go.<br>
<br>
A few general thoughts:<br>
<br>
* One rule we have for CMake is to never expose any public SDK of the C++<br>
implementation structures. We want to be able to rewrite them arbitrarily<br>
at any time. Therefore any solution that needs to access the C++<br>
structures must be integrated into CMake upstream and expose functionality<br>
only through other languages or file formats.<br>
<br>
* The cmState infrastructure builds on a "snapshot" design with a goal of<br>
being able to "fork" configuration/generation temporarily and then revert<br>
back, and to be able to re-start configuration from the middle. These<br>
goals may be incompatible with any language whose implementation we do<br>
not fully control unless it is allowed to execute only in isolated and<br>
independent snippets. These are not hard goals, but it is a trade-off<br>
to keep in mind. Stephen may be able to elaborate more on the snapshot<br>
approach if needed.<br>
<br>
* A problem with the current design is that the entire configuration process<br>
is logically serial making parallel evaluation hard or impossible. In<br>
many cases each add_subdirectory can be processed independently, but this<br>
will require semantic changes to allow.<br>
<br>
On 01/04/2016 02:41 AM, Charles Huet wrote:<br>
> I'm trying to be as declarative as possible, because really like how readable<br>
> simple QML programs are, and I think it would be perfect for a buildsystem.<br>
<br>
Ideally most of the specification (sources, libraries, executables, etc.)<br>
should be in a pure format that can be evaluated without side effects (e.g.<br>
declarative or functional). This rules out both Python and Lua, but the<br>
specification format does not have to be the main entry point. There could<br>
be some imperative configuration step that does system introspection and<br>
then loads the pure specification and evaluates it as needed for the specific<br>
environment.<br>
<br>
If we're going to go through the effort to provide an alternative input format,<br>
I think we should strive for this approach because it will be more flexible in<br>
the long run. A pure specification format will allow easy loading/saving by<br>
other tools, IDEs, etc., without having to process any imperative logic.<br>
<br>
> Actually, I'm directly using the cmMakefile, because I did not want to wrap all<br>
> the commands, and it seemed backwards to me to wrap them.<br>
<br>
Yes. Any alternative format should be processed directly into the structures<br>
used by the generators. The cmState work has separated the generate-time<br>
representation quite a bit from the configuration-time (cmake-language-specific)<br>
representation, but I think there is still further work needed to finish that.<br>
<br>
>> Having said all that, Brad favors Lua I believe, and he favors a different<br>
>> approach (which no one is working on as far as I know) to adding a new<br>
>> language. So wait to hear from him to know whether it is something that<br>
>> would be upstreamable.<br>
><br>
> Have any details on the approach in question?<br>
<br>
See above. Lua has come up several times in the past in particular because<br>
its implementation is meant to be small and embeddable. I've thought a few<br>
times about how to make Lua scripting available from within the CMake language<br>
in a clean way, but that will not be as valuable as the above pure-spec approach.<br>
<br>
> Here is what my test POC looked like for generating a simple shared library:<br>
><br>
> #!/usr/bin/env python<br>
> # -*- coding: utf-8 -*-<br>
> importcmake<br>
> cmake.init("Ninja","/media/dev/src/cmaketest","/media/dev/build/cmaketest")<br>
> myProject=cmake.Project("MyTestProject")<br>
> myProject.targets=[cmake.SharedLibrary("testLibrary",["lib.cxx"])]<br>
> cmake.generate()<br>
<br>
I do not think we should have the build specification depend on processing<br>
code like this. It is not compatible with cmake-gui where the configuration<br>
and generation steps are triggered by the user. However, this does serve<br>
as a good POC that we can populate the generator structures with another<br>
language.<br>
<br>
In summary, I think work in this direction should first focus on designing<br>
a declarative (or functional) specification format where most of the project<br>
information can be specified. Then a cmake-language command can be written<br>
to load and evaluate a specification file (as a transition). Finally we<br>
could look at replacing the entry-point language with something else. At<br>
that point we could have closures passed as parameters to the evaluation of<br>
the pure spec in order to get custom generate-time logic.<br>
<br>
Thanks,<br>
-Brad<br>
<br>
--<br>
<br>
Powered by <a href="http://www.kitware.com" rel="noreferrer" target="_blank">www.kitware.com</a><br>
<br>
Please keep messages on-topic and check the CMake FAQ at: <a href="http://www.cmake.org/Wiki/CMake_FAQ" rel="noreferrer" target="_blank">http://www.cmake.org/Wiki/CMake_FAQ</a><br>
<br>
Kitware offers various services to support the CMake community. For more information on each offering, please visit:<br>
<br>
CMake Support: <a href="http://cmake.org/cmake/help/support.html" rel="noreferrer" target="_blank">http://cmake.org/cmake/help/support.html</a><br>
CMake Consulting: <a href="http://cmake.org/cmake/help/consulting.html" rel="noreferrer" target="_blank">http://cmake.org/cmake/help/consulting.html</a><br>
CMake Training Courses: <a href="http://cmake.org/cmake/help/training.html" rel="noreferrer" target="_blank">http://cmake.org/cmake/help/training.html</a><br>
<br>
Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html" rel="noreferrer" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="http://public.kitware.com/mailman/listinfo/cmake-developers" rel="noreferrer" target="_blank">http://public.kitware.com/mailman/listinfo/cmake-developers</a><br>
</blockquote></div><br></div>
--<br>
<br>
Powered by <a href="http://www.kitware.com" rel="noreferrer" target="_blank">www.kitware.com</a><br>
<br>
Please keep messages on-topic and check the CMake FAQ at: <a href="http://www.cmake.org/Wiki/CMake_FAQ" rel="noreferrer" target="_blank">http://www.cmake.org/Wiki/CMake_FAQ</a><br>
<br>
Kitware offers various services to support the CMake community. For more information on each offering, please visit:<br>
<br>
CMake Support: <a href="http://cmake.org/cmake/help/support.html" rel="noreferrer" target="_blank">http://cmake.org/cmake/help/support.html</a><br>
CMake Consulting: <a href="http://cmake.org/cmake/help/consulting.html" rel="noreferrer" target="_blank">http://cmake.org/cmake/help/consulting.html</a><br>
CMake Training Courses: <a href="http://cmake.org/cmake/help/training.html" rel="noreferrer" target="_blank">http://cmake.org/cmake/help/training.html</a><br>
<br>
Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html" rel="noreferrer" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="http://public.kitware.com/mailman/listinfo/cmake-developers" rel="noreferrer" target="_blank">http://public.kitware.com/mailman/listinfo/cmake-developers</a></blockquote></div></div></div>