[cmake-developers] Generating buildsystem metadata from CMake

[Stephen Kelly]

Tobias Hunger wrote:

>>> and
>>> which kit (think set of compiler, Qt, some other settings) were used
>>> to in the built in that directory.
>>
>> Is this hypothetical? Currently I can add a compiler location and a Qt
>> location in QtCreator, and I can create a kit and give it a name and
>> partcular compiler and Qt. However, the information I configure there and
>> the kit that I choose is not passed to cmake when creator invokes that,
>> right? CMake will do its own determination of the compiler to use, and
>> will find Qt on its own, right? Unless the user manually sets those as
>> arguments in the 'Run CMake' dialog page.
> 
> We do that for qmake projects at this time.
> 
> Nothing listed here is needed to get the level of cmake support we
> have in Creator right *now*: That works fine already after all. I want
> something better:-) So I bring up all the things I can think of to get
> cmake to the same level that we currently have for qmake.

Right, I see.

>> Or are you thinking of scenarios such as 'the user creates a build
>> outside of Creator and then tries to open it in creator'. Given that 'a
>> kit' is something internal to Creator, there is no way CMake could tell
>> you the kit in that scenario. Is that obvious or am I missing something?
>> I think the best thing you can do there is find out which compiler was
>> used, and what the value of Qt5Core_DIR is, and map that to things you
>> know about available kits to find the match if there is one.
> 
> Yes, that is the same we do for qmake projects. In fact we just create
> a new kit for configurations we had not seen before.

Right. In the case that Creator imports a cmake build dir for a project 
which uses Qt 5, and which is not already configured in Creator, you would 
need the path to qmake for the Qt used by it, because that is what Creator 
uses as the key for a particular Qt (or that used to be the case). 

By the way, just as another tip, in that case the reliable way to get the 
corresponding qmake would be to get the Qt5Core_DIR from the cache in the 
just-found build dir (via the json or whatever), and write a file to a temp 
dir with the content:

 cmake_minimum_required(VERSION 2.8)
 project(find_qmake NONE)
 find_package(Qt5Core REQUIRED)
 get_target_property(loc Qt5::qmake LOCATION)
 message("QMAKE_LOCATION: \"${loc}\"")

and then run cmake on it and parse the result:

 stephen at hal:/tmp/cmake/build$ cmake .. \
   -DQt5Core_DIR=/usr/lib/x86_64-linux-gnu/cmake/Qt5Core/
 QMAKE_LOCATION: "/usr/lib/x86_64-linux-gnu/qt5/bin/qmake"
 -- Configuring done
 -- Generating done
 -- Build files have been written to: /tmp/cmake/build

 stephen at hal:/tmp/cmake/build$ cmake .. \
   -DQt5Core_DIR=/opt/qt/binary/Qt5.2.1/5.2.1/gcc_64/lib/cmake/Qt5Core
 QMAKE_LOCATION: "/opt/qt/binary/Qt5.2.1/5.2.1/gcc_64/bin/qmake"
 -- Configuring done
 -- Generating done
 -- Build files have been written to: /tmp/cmake/build

>> I only want the IDE to re-generate when I'm actually working in the IDE.
>> If I edit any file and hit Ctrl+B, the IDE runs 'cmake --build .' and
>> cmake regenerates if needed at that point. It's 'ok' if the project tree
>> is technically 'stale' after I edit a file and before I hit Ctrl+B. It's
>> 'ok' in the sense that the alternative is 'worse' as far as my opinion
>> goes. You also have the option of running 'cmake .' any time any file is
>> saved in the IDE, if that improves the experience, but I strongly believe
>> you shouldn't be watching the files for outside changes and re-generating
>> in response.
> 
> That might be ok for you, but we do get a lot of bug reports whenever
> we have the project tree not update directly. 

'Update directly' means 'I save a file in the IDE and the target treeview 
updates', right? 

Quoting myself above: "running 'cmake .' any time any file is saved in the 
IDE".

> So it is *not* ok for a
> large number of users, particularly those that come from a proprietary
> IDE that comes with a built-in build system.

For my curiosity, what scenario are you talking about which is not covered 
by triggering regeneration on 'saving in the IDE' and 'newly getting focus'?

> We do *not* update creator while it is not in use. 

Fantastic!

> When the window
> gets focus again we go "Things have changed outside Creator, do you
> want to update?".

Great.

>> That's not to say we won't 'give you the rope'. But before we would do
>> that, cmake would have to be defensive against concurrent runs in the
>> same build dir.
> 
> A simple lock file should suffice. I would be surprised if that was
> not there already, but I never checked.

Yes, I think you're right. I haven't checked yet either, but either way it 
is necessary.

>> The key difference is that my proposal is only requested values, so is
>> minimal.
> 
> Yes. But then you need to add "cmake -E list_cache" that lists all
> values. Or how can I find out which values are there in a project?

Indeed.

> In the end it is probably easier to either put all that stuff into the
> json file or to declare CMakeCache.txt to be a stable read-only file
> for the configuration.

Indeed.

>> As you suggest, we could write all of the cache data into the generated
>> cmake-metadata.json file. Alternatively, we could design the interface
>> such that you specify not only what version of the metadata you want, but
>> also which cache values
>>
>>  $ cmake . -DCMAKE_GENERATE_METADATA=3.3 -DCMAKE_METADATA_CACHE="Foo;Bar"
>>
>> to generate a cache section in the metadata file with the 'Foo' and 'Bar'
>> key-value pairs.
>>
>> Is there any particular reason you want "everything" instead of
>> specifying what you want?
> 
> Mostly because I want to be able to rm -rf "builddir" and still be
> able to build:-)

Are we talking about this scenario:

1. I build a project with "Unix Makefiles" generator on the command line.
2. I open the project in Creator - creator reads and caches all values in 
the cache, and I do some work in creator.
3. I decide I want to use the Ninja generator instead. I go an rm -rf on the 
command line.
4. I alt-tab back to creator and it immediately re-populates the build dir I 
just cleared by invoking `cmake . -D$KEY=$VALUE` for every single cache 
entry (therefore ensuring that it stays using "Unix Makefiles").
5. I look at my build dir that I thought I just cleared and wonder what is 
going on.

?

If the build dir gets rm -rf'd, then you should probably treat that in the 
same way as the 'I want to open a project and don't ask me for a build dir' 
scenario discussed below which requires a temp directory.

> I would really appreciate getting all possible information from CMake
> mostly since I doubt you can decide what each and every IDE will need
> that wants to integrate cmake.

Right.

> Please do not try to second-guess which information might be relevant
> to any IDE, that is bound to fail.

I'm trying to find out what is really needed so that we can define stable 
interfaces that don't paint cmake into a corner for future refactorings.

>> Adding a way to make cmake dump the metadata to stdout would be possible,
>> but it wouldn't help...
> 
> Why not? We could set a temporary build dir and you give us the data you
> can.

If you set a temporary build dir, then read the metadata from there. No need 
for stdout reading. Just read the file as normal.

> We have people that want to look into projects and moan when we ask
> them to configure a build dir since they do not understand why they
> need that: They want to browse the code only.

Then choose a temp dir in the background as both of us suggested :).

Quoting myself below:

>> ... CMake needs a directory to run in. It needs to generate files to
>> compile to test the compiler features, determine the platform etc. The
>> content of the project depends on the outcome of those configure-time
>> tests. You need to give cmake a temp directory in this scenario *anyway*.
>> A temp directory from your OS is fine, and you can read the metadata file
>> from there.


> We do need the information from the build system to set up the code
> model correctly and thus need to put them through that -- from their
> point useless -- setup.

Well, you can delay the 'useless setup', but eventually you'll have to ask 
them for a build dir, right?

Thanks,

Steve.