[CMake] CmakeLists chain cannot find source added with add_custom_command

Wed Jul 5 07:08:34 EDT 2017

On Wed, Jul 5, 2017 at 4:05 AM, J Decker <d3ck0r at gmail.com> wrote:

> Was more of a style thing... I had (from another project) a cmakelists
> includable script that builds an amalgamation of sources (makes the
> compilation much faster overall because it doesn't have to read megs of
> headers for windows.h et al. for 50 sources but then only 1.) so I was
> updating to use this script, I already had a folder 'portable' which did
> more of a source based build instead of pulling various DLLs.... so it was
> like
> .../loader (the main project, that links to dlls)  included common
> amalgamation maker here
> .../loader/portable  (the static builds of the same sources) that really
> only contains a single CMakeLists.txt, so rather than use the sources,
> updated to use the amalgamation instead, and it blew up.
>
> I ended up just moving the portable/CMakeLists.txt to
> CMakeLists.portable.txt and including that instead.
>
> Just caught me off guard that it would keep custom targets private to a
> directory instead of making it globally available.  Which would complicate
> implementing this more in lots of other 'portable' (single exe, no DLL
> builds)  But I guess I could create a target that's that amalgamation
> compiled to a static library that could then be referenced by other
> projects.
>

hmm that won't work, the source has the target name compiled into it, so it
would really have to be recompiled anyway....  and various other definition
options might be applied...

>
>
> On Wed, Jul 5, 2017 at 3:34 AM, Eric Noulard <eric.noulard at gmail.com>
> wrote:
>
>> OK right.
>> I missed the proper FAQ entry, thanks Nicholas for pointing it out.
>>
>> Eric
>>
>> 2017-07-05 12:10 GMT+02:00 Nicholas Devenish <ndevenish at gmail.com>:
>>
>>> Hi,
>>>
>>> It looks like this is the issue covered in
>>> https://cmake.org/Wiki/CMake_FAQ#How_can_I_add_a_dependency_
>>> to_a_source_file_which_is_generated_in_a_subdirectory.3F
>>> and bug https://gitlab.kitware.com/cmake/cmake/issues/14633 ? I
>>> suppose if you are writing your own systems from scratch it's easy not
>>> to run into this, but a lot easier when converting existing
>>> complicated build systems.
>>>
>>> I kind of agree that it's easy to miss this constraint reading the
>>> documentation, but there doesn't seem to be a better suggested
>>> solution yet.
>>>
>>> Nick
>>>
>>>
>>> On Sun, Jul 2, 2017 at 11:57 AM, Eric Noulard <eric.noulard at gmail.com>
>>> wrote:
>>> > Ok,
>>> >
>>> > I reproduced your issue and I can then confirm it.
>>> > I tried several thing including:
>>> >    - generating the file is the buildtree
>>> >    - refering to the same file with another target in the same dir
>>> >    - using absolute path in the sub2/add_executable
>>> >
>>> > nothing works.
>>> > it looks like a generated file cannot be referred outside the directory
>>> > where the custom command generates it ???
>>> >
>>> > Here comes an archive with an example similar to the one descrived by
>>> J.
>>> > Decker.
>>> > If you build it without option it wokrs (because I used an intermediate
>>> > object lib)
>>> > If you (try to) build it with
>>> >
>>> > cmake -DBROKEN=1
>>> >
>>> > then it'll fails with "Cannot find source file..."
>>> >
>>> > This is definitely weird and I never encountered this after +10 years
>>> of
>>> > CMake usage!!
>>> > So I guess I have always used generated file locally to build exe or
>>> lib and
>>> > then refer to the lib.
>>> > If this is the intended behaviour then it should be documented more
>>> > explicitely.
>>> >
>>> > Now re-reading the doc of add_custom_command:
>>> >
>>> > "This defines a command to generate specified ``OUTPUT`` file(s).
>>> > A target created in the same directory (``CMakeLists.txt`` file)
>>> > that specifies any output of the custom command as a source file
>>> > is given a rule to generate the file using the command at build time."
>>> >
>>> > So the "same directory" should be underlined!! Or may be another
>>> sentence
>>> > saying that in fact you CANNOT refer to outputed files outside the
>>> > directory...
>>> >
>>> > In fact it is possible to have cross-directory reference to generated
>>> file
>>> > but this is
>>> > more like a workaround. You'll have to:
>>> >
>>> > define your custom_command and an associated custom target in dir1
>>> > define your exe/lib in dir2 and explicitely:
>>> >    - flag the referred cross-dir generated source as GENERATED
>>> >    - make your exe/lib depend on the cross-dir custom target in order
>>> to
>>> > trigger the file generation.
>>> >
>>> >
>>> > The second zip implement that workaround. This is nasty, I wouldn't do
>>> that.
>>> > Shall we open a bug for this misleading behavior or shall we open a
>>> bug for
>>> > doc update?
>>> >
>>> >
>>> >
>>> > 2017-07-02 10:20 GMT+02:00 J Decker <d3ck0r at gmail.com>:
>>> >>
>>> >> It is not generated; because it is generated during a the build phase
>>> not
>>> >> the generate phase.  So only the logical idea of the file exists.
>>> >>
>>> >> On Sun, Jul 2, 2017 at 12:12 AM, Eric Noulard <eric.noulard at gmail.com
>>> >
>>> >> wrote:
>>> >>>
>>> >>> Needless to say that your example should work but somehow CMake does
>>> not
>>> >>> see that the referred file is GENERATED.
>>> >>>
>>> >>> Le 2 juil. 2017 09:11, "Eric Noulard" <eric.noulard at gmail.com> a
>>> écrit :
>>> >>>>
>>> >>>> Is the referred file created?
>>> >>>> Does your actual command creating the file doing it in the source
>>> tree
>>> >>>> and not in the build tree?
>>> >>>> Are you building in-source or out of source?
>>> >>>>
>>> >>>> Could you give us us a toy example which shows the issue?
>>> >>>>
>>> >>>> Le 2 juil. 2017 04:32, "J Decker" <d3ck0r at gmail.com> a écrit :
>>> >>>>
>>> >>>> Known issue?  Or just doesn't require a response?
>>> >>>>
>>> >>>> On Wed, Jun 28, 2017 at 6:05 AM, J Decker <d3ck0r at gmail.com> wrote:
>>> >>>>>
>>> >>>>> I have this chain of makefiles.  It adds a custom rule to build a
>>> >>>>> source file, and then a subdirectory cannot reference that file.
>>> >>>>>
>>> >>>>> The first is added custom rule for M:/tmp/cmake-chain/sub/src/sac
>>> k.c
>>> >>>>> and then later cannot find M:/tmp/cmake-chain/sub/src/sack.c
>>> >>>>> though it's really referencing it as
>>> >>>>> M:/tmp/cmake-chain/sub/sub2/../src/sack.c
>>> >>>>>
>>> >>>>> tested with 3.6.0-rc4 and 3.9.0-rc5
>>> >>>>>
>>> >>>>> if the add_custom_rule and add_executable are in the same makefile
>>> even
>>> >>>>> if there's a added path to make it not exactly the same source as
>>> the
>>> >>>>> 'referencing source' message logs; it works.
>>> >>>>>
>>> >>>>>
>>> >>>>> --- CMakeLists.txt ---
>>> >>>>> cmake_minimum_required(VERSION 2.8)
>>> >>>>>
>>> >>>>> message( "Adding custom rule for
>>> >>>>> ${CMAKE_CURRENT_SOURCE_DIR}/src/sack.c" )
>>> >>>>>   add_custom_command( OUTPUT ${CMAKE_CURRENT_SOURCE_DIR}/sr
>>> c/sack.c
>>> >>>>>   COMMAND cmake -E echo do something to make src/sack.c
>>> >>>>>   )
>>> >>>>> add_subdirectory( sub2 )
>>> >>>>> -------------------------------------
>>> >>>>>
>>> >>>>> --- sub2/CmakeLists.txt ---
>>> >>>>> message( "referencing source ${CMAKE_CURRENT_SOURCE_DIR}/..
>>> /src/sack.c"
>>> >>>>> )
>>> >>>>> add_executable( something ${CMAKE_CURRENT_SOURCE_DIR}/../src/sack.c
>>> )
>>> >>>>>
>>> >>>>> -------------------------------------
>>> >>>>>
>>> >>>>> --- Truncated output log ------
>>> >>>>> Adding custom rule for M:/tmp/cmake-chain/sub/src/sack.c
>>> >>>>> referencing source M:/tmp/cmake-chain/sub/sub2/../src/sack.c
>>> >>>>> -- Configuring done
>>> >>>>> CMake Error at sub2/CMakeLists.txt:3 (add_executable):
>>> >>>>>   Cannot find source file:
>>> >>>>>
>>> >>>>>     M:/tmp/cmake-chain/sub/src/sack.c
>>> >>>>>
>>> >>>>>   Tried extensions .c .C .c++ .cc .cpp .cxx .m .M .mm .h .hh .h++
>>> .hm
>>> >>>>> .hpp
>>> >>>>>   .hxx .in .txx
>>> >>>>> --------------------------------------
>>> >>>>>
>>> >>>>>
>>> >>>>> ------ Flat CmakeLists.txt which works ---------------
>>> >>>>> cmake_minimum_required(VERSION 2.8)
>>> >>>>>
>>> >>>>> message( "Adding custom rule for
>>> >>>>> ${CMAKE_CURRENT_SOURCE_DIR}/src/sack.c" )
>>> >>>>>   add_custom_command( OUTPUT ${CMAKE_CURRENT_SOURCE_DIR}/sr
>>> c/sack.c
>>> >>>>> COMMAND cmake -e echo make file.
>>> >>>>>   )
>>> >>>>>
>>> >>>>> message( "referencing source ${CMAKE_CURRENT_SOURCE_DIR}/src/sack.c"
>>> )
>>> >>>>> add_executable( something
>>> >>>>> ${CMAKE_CURRENT_SOURCE_DIR}/sub2/../src/sack.c )
>>> >>>>> --------------------------------------
>>> >>>>>
>>> >>>>> ----- flat cmakelists output.... -----------
>>> >>>>> Adding custom rule for M:/tmp/cmake-chain-flat/src/sack.c
>>> >>>>> referencing source M:/tmp/cmake-chain-flat/sub2/../src/sack.c
>>> >>>>> -- Configuring done
>>> >>>>> --------------------------------------
>>> >>>>>
>>> >>>>
>>> >>>>
>>> >>>> --
>>> >>>>
>>> >>>> Powered by www.kitware.com
>>> >>>>
>>> >>>> Please keep messages on-topic and check the CMake FAQ at:
>>> >>>> http://www.cmake.org/Wiki/CMake_FAQ
>>> >>>>
>>> >>>> Kitware offers various services to support the CMake community. For
>>> more
>>> >>>> information on each offering, please visit:
>>> >>>>
>>> >>>> CMake Support: http://cmake.org/cmake/help/support.html
>>> >>>> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>>> >>>> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>> >>>>
>>> >>>> Visit other Kitware open-source projects at
>>> >>>> http://www.kitware.com/opensource/opensource.html
>>> >>>>
>>> >>>> Follow this link to subscribe/unsubscribe:
>>> >>>> http://public.kitware.com/mailman/listinfo/cmake
>>> >>>>
>>> >>>>
>>> >>
>>> >>
>>> >> --
>>> >>
>>> >> Powered by www.kitware.com
>>> >>
>>> >> Please keep messages on-topic and check the CMake FAQ at:
>>> >> http://www.cmake.org/Wiki/CMake_FAQ
>>> >>
>>> >> Kitware offers various services to support the CMake community. For
>>> more
>>> >> information on each offering, please visit:
>>> >>
>>> >> CMake Support: http://cmake.org/cmake/help/support.html
>>> >> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>>> >> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>> >>
>>> >> Visit other Kitware open-source projects at
>>> >> http://www.kitware.com/opensource/opensource.html
>>> >>
>>> >> Follow this link to subscribe/unsubscribe:
>>> >> http://public.kitware.com/mailman/listinfo/cmake
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Eric
>>> >
>>> > --
>>> >
>>> > Powered by www.kitware.com
>>> >
>>> > Please keep messages on-topic and check the CMake FAQ at:
>>> > http://www.cmake.org/Wiki/CMake_FAQ
>>> >
>>> > Kitware offers various services to support the CMake community. For
>>> more
>>> > information on each offering, please visit:
>>> >
>>> > CMake Support: http://cmake.org/cmake/help/support.html
>>> > CMake Consulting: http://cmake.org/cmake/help/consulting.html
>>> > CMake Training Courses: http://cmake.org/cmake/help/training.html
>>> >
>>> > Visit other Kitware open-source projects at
>>> > http://www.kitware.com/opensource/opensource.html
>>> >
>>> > Follow this link to subscribe/unsubscribe:
>>> > http://public.kitware.com/mailman/listinfo/cmake
>>>
>>
>>
>>
>> --
>> Eric
>>
>> --
>>
>> Powered by www.kitware.com
>>
>> Please keep messages on-topic and check the CMake FAQ at:
>> http://www.cmake.org/Wiki/CMake_FAQ
>>
>> Kitware offers various services to support the CMake community. For more
>> information on each offering, please visit:
>>
>> CMake Support: http://cmake.org/cmake/help/support.html
>> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/cmake
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cmake/attachments/20170705/a8de2cbb/attachment-0001.html>