[cmake-developers] Extracting target metadata, IDE integration

[Aleix Pol]

Hi Tobias,
Thank you for taking some of your time.

On Sat, Mar 7, 2015 at 11:46 AM, Tobias Hunger <tobias.hunger at gmail.com> wrote:
> Hi Aleix,
>
> thank you for the ping:-)
>
> Since I am not fully fluent with cmake (have not really used it in a
> couple of years) and sometimes have a bit of trouble following the
> discussion about the details of this patch, I took the liberty to
> copy/paste (and part of) one of the example files you put up on KDEs
> codepaster, commenting on what I understood each section to mean. I
> think that is the best way for me to contribute to this discussion at
> this point.
>
> What are you actually building here? /home/kde-devel/tmp/llvm/CMakeLists.txt?
Thats an LLVM code clone. It's a big CMake-based project, that's why I
chose it, that's all.

>
>> {
>
> How about adding some information on what this is you are actually
> building here? CMake has the PROJECT command that does provide quite a
> bit of information that could go here.
I can look into that, I can see if I can extract the projects tree
somehow and dispose them like it's doing with the targets.

>
> Where are the sources this thing here is building? How were those
> sources configured? CMake supports quite a few generators, so which
> ones where used? Do I need to run ninja or make to build this (or
> should I use "cmake build" anyway)?
The sources are listed within the targets, separately for each
configuration (as you can have different sources for debug builds and
release).

Regarding ninja/make, in KDevelop we just check if there's a
rules.ninja file or a Makefile. Alternatively you can also check the
CMAKE_GENERATOR value in the CMakeCache.txt. Or we can introduce it if
you find it useful. I don't have a strong opinion there.

>
> Creator does allow existing qmake builds to be imported and that kind
> of information does help a lot with that functionality.
>
>> "dependencies" :
>> [ "/home/kde-devel/tmp/llvm/build/CMakeFiles/3.2.20150306-g139588/CMakeSystem.cmake", "/lots/more/files", ... ]
>
> So these are all the files I need to watch for changes? If they change
> then I should re-run cmake?
Yes, also they are the files that will be providing the relevant
information for the current file. It can be important at some point.

>
>> "targets" :
>> [
>
> So now comes a list of the possible things I can build...
>
>>   {
>>     "backtrace" :
>>     [
>>       "/home/kde-devel/tmp/llvm/cmake/modules/AddLLVM.cmake:312",
>>       "/home/kde-devel/tmp/llvm/cmake/modules/AddLLVM.cmake:399",
>>       "/home/kde-devel/tmp/llvm/lib/Target/PowerPC/TargetInfo/CMakeLists.txt:1"
>>     ],
>
> That backtrace is nice!
>
>>    "configs" :
>>    [
>>        {
>>            "name" : "",
>
> Why is the name empty?
Because it was built without specifying any name. (i.e. it didn't pass
-DCMAKE_BUILD_TYPE=...).

>
>>            "output" : "/home/kde-devel/tmp/llvm/build/lib/libLLVMPowerPCInfo.a",
>
> Can this be a list?
It's 1 output per target, no?

>
>>            "sources" : [ "/list/of/cpp/files, ]
>
> Any way to also get the headers? Do you see any option to hint where
> new sources would need to be added to make them a part of this target?
I looked into how the CodeBlocks did it to provide the header files.
It's just brute-forcing to see if the file renamed to *.h/*.hpp/*.hxx
is present.
CMake doesn't have this kind of information, so I think it's better
that the IDE's do the hacky parts than cmake.
Also we don't really have information about where to put them. You can
use the minimum common denominator among the sources I guess.

>
>>        }
>>    ],
>>    "installed" : "true",
>
> I guess this means "make install" will install it. Where will this file end up?
Yes. I tried to figure it out but didn't manage to find the correct
way to query it. I can try harder. :D

>
>>    "name" : "LLVMPowerPCInfo",
>>    "output_directory" : "/home/kde-devel/tmp/llvm/build/lib/Target/PowerPC/TargetInfo",
>
> For what is the output directory relevant? The output above does
> implicitly list that already, doesn't it?
It's important in case the output path is overriden, see last e-mail
by Alexander Neundorf.

>
>>    "type" : "STATIC_LIBRARY"
>
> Which types are possible here?
It's an enum internal to cmake. At the moment:
EXECUTABLE, STATIC_LIBRARY, SHARED_LIBRARY, MODULE_LIBRARY,
OBJECT_LIBRARY, UTILITY, GLOBAL_TARGET, INTERFACE_LIBRARY,
UNKNOWN_LIBRARY.

>
>> },
>
>
> There is a lot of useful information here! Thanks for pushing into
> this direction. I am sure this will help a lot.
>
> Ideally this file would be enough to provide all information we need
> as an IDE to:
>
> * Present the project structure as seen by the build system.
> * Generate the code model
> * Build the project
> * Deploy the project
> * Run and debug parts of the project
> * Adding and removing files to a project
>
> For the project structure this does help: It provides us with a list
> of all the build targets, which is already nice. CMake does support
> more structure with Projects and subprojects IIRC and that information
> is lost. Would it be much work to get that information back?
as discussed above, I'll look into that next thing.

>
> For the code model this is not much help at all: There is no
> information on compiler used, include directories nor Defines or
> compiler flags. Where am I supposed to get that information from? Do I
> need to generate and parse the list of commands run and extract that
> information from there? Having that information available right in the
> sources array of each target would be so much more convenient and
> would also make IDE integration so much easier: You would need to know
> about one file only and won't need to figure out several.
We can do that, I mentioned earlier in this thread that currently
we're using compile_commands.json for that (
CMAKE_EXPORT_COMPILE_COMMANDS).

I agree it could be interesting to add it. In other words, if nobody
disagrees, I'll add it although I see how this will make the output
grow a lot.

>
> Building the project should be fine with the information provided.
> Explicitly stating the generators used would help a little bit (even
> better just giving the command necessary to build, e.g. make or ninja
> or whatnot. Then we would not need to hard-code the mapping of
> generators->build tools, which makes it much easier for you to support
> new things:-)
>
> Running/Debugging is a bit tricky again: All the information used for
> linking the binaries is gone. So there is no way to figure out which
> LD_LIBRARY_PATH needs to be set or anything. There also seems to be no
> information where files are going to get installed to. Would it be
> possible to add that information?
I ignore it, on the other hand I'm quite a n00b when it comes to cmake
codebase and internals. I'm sure if somebody knows where to get that
information, he is reading this mailing list. (hello)

>
> I do not see any way to provide a reliable automatic way to add
> sources to a cmake build system (the others are unfortunately not
> better:-). For removing the files it would be nice to know where in
> which file/line the sources were added to the target. Not sure whether
> it is even possible to get that information... The backtrace you have
> is already great piece of information: At least that way we should be
> able to open the right file and have the user edit that manually.
Well, what I did in kdevelop 4 (before adopting all of this) was going
to the last CMakeLists.txt on the backtrace and looking for the
filename in the arguments there (and recursively through the variable
definitons, which we don't have the information in here) and removing
it. It usually worked and when it didn't, we just didn't know what to
do, but it wasn't essentially wrong. You don't want file references of
a file you're removing anyway.

>
> Does this help?
Yes, a lot. It's what I was looking for.

>
> Best Regards,
> Tobias

Thanks for taking your time.

Aleix