[CMake] Dependency cycle - why?

[Bill Somerville]

Hi Eric,

thanks for the comprehensive reply to my query. My follow up comments in 
line below.

On 25/05/2019 15:47, Eric Noulard wrote:
> Le sam. 25 mai 2019 à 13:51, Bill Somerville <bill at classdesign.com 
> <mailto:bill at classdesign.com>> a écrit :
>
>     Hi Robert,
>
>     thanks for that, the target name change does seem to help but I am
>     still unable to achieve my goal. Here is a simplified example that
>     demonstrates the problem:
>
>     cmake_minimum_required (VERSION  3.1.0 FATAL_ERROR)
>     project (demo LANGUAGES NONE)
>     add_custom_target (prog_target COMMAND ${CMAKE_COMMAND} -E touch prog${CMAKE_EXECUTABLE_SUFFIX})
>     add_executable (prog IMPORTED)
>     add_dependencies (prog prog_target)
>     set_target_properties (prog PROPERTIES IMPORTED_LOCATION ${CMAKE_CURRENT_BINARY_DIR}/prog${CMAKE_EXECUTABLE_SUFFIX})
>     install (TARGETS prog RUNTIME DESTINATION bin)
>
>     which gives the following error at CMake configuration:
>
>     CMake Error at CMakeLists.txt:7 (install):
>        install TARGETS given target "prog" which does not exist.
>
>
>     -- Configuring incomplete, errors occurred!
>
>     So the target that 'add_executable(name IMPORTED)' creates is not
>     a real executable target. I can change its properties but the
>     'install(TARGETS ...)' command thinks it does not exist. Note that
>     a executable target is a very simple demonstration and I
>     understand that I can use 'install(PROGRAM ...)' just about as
>     easily, but when it comes to a shared library it gets a lot more
>     complex when using, exporting, and instlling it, and it seems that
>     IMPORTED targets fall well short of useful when they are actually
>     produced by the current CMake project.
>
>     I can understand that an IMPORTED target is perhaps not meant to
>     be installable
>
>
> Robert will give his advice on that but I bet IMPORTED target were 
> never meant to be drop-in *replacement* of genuine target.
>
> They were meant to ease *reference to* object/lib:executable that are 
> outside the build of the project.
> e.g the doc says: 
> https://cmake.org/cmake/help/latest/command/add_library.html#imported-libraries
> "An IMPORTED library target references a library file located outside 
> the project"
>
> Nmelly a target that is "already installed somewhere" and that you 
> want to reference in your CMake build.
That is fine but I don't see why there is a problem with making IMPORTED 
targets better CMake citizens, it should be down to the CMake script 
author as to whether they take advantage of that. What is there now is a 
lie and the add_*(target ... IMPORTED ...) commands are not doing what 
they should IMHO.
>
>     but if so then your statement that "The goal that you have is
>     fully supported by CMake" seems to be incorrect. To reiterate, I
>     am trying to use foreign tools to make binary targets and wish to
>     have CMake treat them *as if* they were created by supported
>     languages like C, ++, or Fortran. Am I still missing something?
>
>
> My opinion is that IMPORTED target are not designed (as of today) for 
> that purpose.
Agreed.
>
> When you want to do what you seem to want you need to either:
> 1) add a "new LANGUAGE" (you seem to be willing to add golang)
>     see: 
> https://stackoverflow.com/questions/7978517/how-do-i-get-cmake-to-work-with-the-go-programming-language
>     and may be: https://github.com/aadidenko/golang-cmake-example (or 
> fork of this)
I am not sure that adding a new CMake supported language is right for 
Go. For the Go ecosystem the Go build tools do the right thing and CMake 
is largely unnecessary. Particularly the Go build system deals with 
dependencies internally so I don't see how it could be added to CMake 
without using the lower level Go tools and having to list many source 
dependencies that the high level Go build system deals with without 
fuss. Note that I am dealing with cgo which allows Go language static or 
shared libraries to interoperate with other languages via a C API. This 
is where CMake needs to join in because a project using cgo is likely to 
build tests, tools, and libraries in languages that CMake does support. 
As far as I can see the Go build system support is weak in providing a 
"package" from, for example, a cgo shared library, it just builds the 
shared object and header files and stops there.
>
> 2) define a set of custom macros that mimic genuine target behaviour.
>     may be see: https://github.com/cpconduce/go_cmake
This is exactly where I started but it only covers tools written in Go 
and uses install(PROGRAMS ...) to install them. It uses 
add_custom_command() which means all source dependencies must be 
specified for reliable rebuilds, which is effectively discarding the 
dependency management of the Go build system. It would be very fragile 
in a large project and doesn't address the cgo capabilities to create 
libraries. My queries here are exactly because I am trying to extend 
this approach to tools and libraries that play well in the CMake 
ecosystem, without sacrificing the advantages of the Go build system. I 
can, and have, written CMake functions that use add_custom_target() to 
defer Go builds to the Go build system, but am failing so far to bridge 
the results with CMake. Clearly at present IMPORTED targets do not work 
for this, so I guess I am asking how to do this another way, e.g. by 
setting all the necessary target properties to make my targets proper 
CMake citizens?
>
> If you want to have a look at existing similar example shippped with 
> CMake, have a look at UseJava.cmake module 
> (https://cmake.org/cmake/help/latest/module/UseJava.html)
> You'll see that you have
> add_jar, which is similar to add_executable and create a custom target
> and
> install_jar which is similar to install on genuine target but plays 
> with target properties and install(FILES...) in order to mimic that.
> see: https://github.com/Kitware/CMake/blob/master/Modules/UseJava.cmake
This looks more interesting and complete but the first thing I note is 
that it does not really integrate with the CMake ecosystem, instead it 
has a parallel set of commands (add|install|find|export)_jar() that 
mimic the CMake equivalents. That may work well for Java, even including 
a foreign function interface as it seems to support JNI, but there is a 
key difference. A Java class file is linked purely at runtime so there 
is no need for CMake to know about how to link the JAR file to a native 
executable target (it uses add_library(target INTERFACE ...) to 
integrate the generated API headers). With Go and cgo the generated 
library *is a native library*, just as Go tools are native binaries, and 
to use them seamlessly in CMake they have to behave as any other 
imported or built executable target, a parallel set of CMake commands 
that do not integrate the target will not get the job done.
>
> I'm not developing with golang but AFAIK go has a builtin "build 
> system" so bringing go as a language in CMake may not be the best option,
> but again I am no Go expert.

You are correct and this is exactly why I am looking for a way to hook 
the products of the Go build system into CMake rather than trying to add 
it as a supported language. I do not know enough abut the low level Go 
build system commands to understand if they are suitable for driving 
from CMake like it does for C, C++, or Fortran compilers and the target 
system linker but that seems to be the wrong approach, I'd rather use 
the top level 'go build' command and tell CMake about what I expect it 
to build and what top level targets I want to be up to date before that 
step. This seems to be a simple add_custom_target() invocation. There is 
a small penalty that the 'go build' command will always run but that's 
fine since I expect 'go build' to do minimal re-builds based on its own 
dependency tracking.

The approach I am current trying is to treat cgo libraries as INTERFACE 
libraries and having my CMake scripts reference the library binaries by 
their path by misusing target_link_libraries(so_library_target INTERFACE 
...) to hook things up. This doesn't help with installing the library 
target but that can be done using install(PROGRAMS ...). That at least 
allows install(EXPORT ...) to work, I think - I have not yet tried 
importing the shared library into anther CMake project.

>
> The only reference of that kind of thing I found in the CMake mailing 
> list is oldish:
> https://cmake.org/pipermail/cmake-developers/2011-August/013715.html
That link shows exactly why using the Go build tools as a custom target 
command is right IMHO, it also doesn't address using the product of the 
build in the same or other projects.
>
> May be adding the support for "install" command for IMPORTED target is 
> doable but this seems to be a feature request to be discussed on 
> developer mailing list;

I think there is justification since a CMake script author decides 
whether to take advantage of it. It seems to me that there is stuff that 
is internal to the CMake C++ application similar to properties but not 
addressable by the script author. Of course it all depends how complex 
that stuff is and whether if can be addressed by either some new target 
properties or extra arguments to the add_(executable|library)(target 
IMPORTED) commands. The latter seems easiest as the internal logic and 
data model need not be exposed for misuse by script authors.

If there really is not a practical way to build a library or tool with a 
foreign toolchain in CMake and have it integrated like the ones produced 
by supported languages then I will raise an issue on the development 
forum, but TBH I am disappointed that manipulating existing target 
properties is not sufficient. Custom targets seem to fall well short of 
what they promise.

> Regards,
> Eric
Regards
Bill Somerville.


>
>
>     Regards
>     Bill Somerville.
>
>     On 24/05/2019 20:23, Robert Maynard wrote:
>>     Hi,
>>
>>     The goal that you have is fully supported by CMake. You have just run
>>     into a bug in CMake, and you should report this to
>>     https://gitlab.kitware.com/cmake/cmake/issues  .
>>     Basically at a very high level the name out the add_executable target
>>     `callback_generator` is the same as the internal name that CMake is
>>     using for the add_custom_command. This than causes some logic in CMake
>>     (cmTargetTraceDependencies) to incorrectly build another link between
>>     the add_custom_command and your add_executable.
>>
>>     The easiest way to solve this issue is name the add_executable target
>>     to have a different name than the output of your custom command minus
>>     file extension. So maybe something like  `callback_generator_exec`
>>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://cmake.org/pipermail/cmake/attachments/20190525/7892e8d9/attachment-0001.html>