[CMake] Generated files?
Chris Johnson
cxjohnson at gmail.com
Fri Jan 30 12:09:54 EST 2015
Ah, I found the problem. I had the output file listed in the custom
command's DEPEND clause.
Thank you very much for all the help!
On Fri, Jan 30, 2015 at 11:07 AM, Chris Johnson <cxjohnson at gmail.com> wrote:
> I'm trying to use the method outlined in your #2, with the list of files.
> But I get circular dependency errors, with messages like this one (for each
> file):
>
> make[2]: Circular validate/X12_Standards/997_5010.x12 <-
> validate/X12_Standards/997_5010.x12 dependency dropped.
>
> On Fri, Jan 30, 2015 at 10:28 AM, Petr Kmoch <petr.kmoch at gmail.com> wrote:
>
>> I believe the classic "driving target" approach could be used here. Add a
>> custom target which will drive the custom commands by depending on their
>> outputs:
>>
>> add_custom_target(
>> RunMyProg ALL
>> DEPENDS
>> ${CMAKE_CURRENT_BINARY_DIR}/file001.out
>> ${CMAKE_CURRENT_BINARY_DIR}/file002.out
>> #...
>> COMMENT "Processing files with MyProg"
>> )
>>
>> Notice the ALL after the target name - this makes the target part of
>> 'make all' or equivalent.
>>
>> Custom targets are always considered out of date (always rebuilt), but
>> this one has no COMMAND and so building it does nothing. It does make sure
>> all its dependencies (the .out files) are up to date, though, and if they
>> are not, it will run the rules to build them. These rules are the custom
>> commands you've created earlier.
>>
>> This "custom target driving custom commands" is a very idiomatic pattern
>> in CMake.
>>
>> Of course, you don't want to list the files by hand. You could modify my
>> earlier function in one of two ways:
>>
>> 1. Have each invocation append to a list:
>>
>> set(OutputFileList "")
>>
>> function(ProcessFile inFile)
>> get_filename_component(outFile ${inFile} NAME_WE)
>> set(outFile ${inFile}.out)
>> add_custom_command(
>> OUTPUT ${outFile}
>> COMMAND ${CMAKE_COMMAND} -E copy
>> ${CMAKE_CURRENT_SOURCE_DIR}/${inFile} ${CMAKE_CURRENT_BINARY_DIR}
>> COMMAND myprog ${inFile} ${outFile}
>> DEPENDS ${inFile} myprog.cpp
>> )
>> install(FILES ${CMAKE_CURRENT_BINARY_DIR}/${outFile} DESTINATION
>> share/docs)
>> set(OutputFileList ${OutputFileList}
>> ${CMAKE_CURRENT_BINARY_DIR}/${outFile} PARENT_SCOPE) # This line is new
>> endfunction()
>>
>> # process files
>> ProcessFile(file001.in)
>> ProcessFile(file002.in)
>> ProcessFile(file003.in)
>>
>>
>> # now use the list in the custom target:
>> add_custom_target(
>> RunMyProg ALL
>> DEPENDS ${OutputFileList}
>> COMMENT "Processing files with MyProg"
>> )
>>
>>
>> Or, 2., refactor the function to take a list of files and do everything
>> itself:
>>
>> function(ProcessFiles)
>> set(OutputFileList "")
>> foreach(inFile IN LISTS ARGN)
>> get_filename_component(outFile ${inFile} NAME_WE)
>> set(outFile ${inFile}.out)
>> add_custom_command(
>> OUTPUT ${outFile}
>> COMMAND ${CMAKE_COMMAND} -E copy
>> ${CMAKE_CURRENT_SOURCE_DIR}/${inFile} ${CMAKE_CURRENT_BINARY_DIR}
>> COMMAND myprog ${inFile} ${outFile}
>> DEPENDS ${inFile} myprog.cpp
>> )
>> list(APPEND OutputFileList ${CMAKE_CURRENT_BINARY_DIR}/${outFile})
>> endforeach()
>> install(FILES ${OutputFileList} DESTINATION share/docs)
>> add_custom_target(
>> RunMyProg ALL
>> DEPENDS ${OutputFileList}
>> COMMENT "Processing files with MyProg"
>> )
>> endfunction()
>>
>> # now just call it once with the list of files:
>> ProcessFiles(
>> file001.in
>> file002.in
>> file003.in
>> )
>>
>>
>> You can naturally enlist file(GLOB) to grab the list of files, as J
>> Decker suggests. The downside, as with any list of source files, is that
>> CMake will not know that it needs to re-run when you simply add a new file
>> which matches the glob. If you list the files explicitly, you have to edit
>> the CMakeList to add the file, and CMake will know to re-run at next build.
>>
>> Petr
>>
>>
>>
>>
>>
>> On Fri, Jan 30, 2015 at 4:40 PM, Chris Johnson <cxjohnson at gmail.com>
>> wrote:
>>
>>> Petr, thanks. That makes a lot of sense.
>>>
>>> Now I am struggling with causing all of these files to be generated when
>>> I do the default build target, e.g. just "make" in the build directory.
>>> That is, I want myprog to be compiled, and then all of the files processed
>>> by it with just one build command. I assume that's possible, but have not
>>> quite yet been able to make it happen.
>>>
>>> On Fri, Jan 30, 2015 at 9:11 AM, Petr Kmoch <petr.kmoch at gmail.com>
>>> wrote:
>>>
>>>> As you say, **make** is fairly powerful at this. Other build tools may
>>>> not be, it might even be an alien concept to them. AFAIK, Ninja requires an
>>>> explicit list of files & dependencies, for example. Visual Studio solutions
>>>> can only contain explicitly listed files too.
>>>>
>>>> CMake projects are supposed to work with all generators which CMake
>>>> supports (within reason). There are some generator-specific features in
>>>> CMake, but the intent is to keep the set of these to a minimum.
>>>>
>>>> The list of CMake commands which directly create build-time "commands"
>>>> (such as Makefile rules) is rather short:
>>>>
>>>> add_custom_command()
>>>> add_custom_target()
>>>> add_executable()
>>>> add_library()
>>>>
>>>> Other commands modify the buildsystem (e.g. add_definitions(),
>>>> target_link_libraries(), or setting properties such as COMPILE_OPTIONS),
>>>> but they don't really add "commands" to it.
>>>>
>>>> Petr
>>>>
>>>>
>>>> On Fri, Jan 30, 2015 at 3:27 PM, Chris Johnson <cxjohnson at gmail.com>
>>>> wrote:
>>>>
>>>>> That does clarify a few things yes. Thank you.
>>>>>
>>>>> By "configure" time I had meant the first two items you listed as
>>>>> Configure and Generate. By "run" time I had meant what you called Build.
>>>>>
>>>>> One place I am unclear on is exactly which commands in CMakeLists.txt
>>>>> get translated into Build time commands in buildsystem files (in my case,
>>>>> make). For example, make is fairly powerful at generating substitution
>>>>> file names, e.g. creating prog.o from prog.cpp, or file1.out from
>>>>> file1.in. Make is also capable of running full shell scripts within
>>>>> a makefile. But there's no obvious way to drive any kind of automation
>>>>> from CMake that make is capable of.
>>>>>
>>>>> As a programmer, my first thought when having to process a large list
>>>>> of files is to do it through some sort of automation, rather than generate
>>>>> a brute force solution. It appears that with CMake, the best one can hope
>>>>> for is to automate the creation of the brute-force solution.
>>>>>
>>>>> --
>>>>> Cheers,
>>>>> ..chris
>>>>>
>>>>> On Fri, Jan 30, 2015 at 4:32 AM, Petr Kmoch <petr.kmoch at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> It's not clear to me what you mean with "run" time. There are the
>>>>>> following "times" involved in a CMake-based project:
>>>>>>
>>>>>> * Configure time:
>>>>>> CMake is parsing CMakeLists.txt files and files included from those
>>>>>> and executing ALL CMake commands found in them. Variables expansion takes
>>>>>> place. Data structures in CMake's memory are filled.
>>>>>>
>>>>>> * Generate time:
>>>>>> CMake processes the data structures in its memory to write
>>>>>> buildsystem files (Makefiles, solutions etc.) to disk. Generator
>>>>>> expressions are expanded. Files are generated from data specified in
>>>>>> file(GENERATE ...) commands. cmake_install.cmake file is written based on
>>>>>> the contents of data structures filled by install() commands.
>>>>>>
>>>>>> * Build time:
>>>>>> CMake is not running (except for driving the build if you do that
>>>>>> with `cmake --build`). The build tool (make, ninja, devenv, msbuild, ...)
>>>>>> is running, processing the buildsystem files and generating compiled
>>>>>> binaries. At this time, commands which were set up by add_custom_command()
>>>>>> and add_custom_target() are executed as driven by the build tool.
>>>>>>
>>>>>> * Install time (optional):
>>>>>> Running `make install` or similarly triggering the install process
>>>>>> causes CMake to run in installation mode. This CMake reads the
>>>>>> cmake_install.cmake file and executes the commands there. These are mainly
>>>>>> file() commands which copy files to the desired location.
>>>>>>
>>>>>> CMake commands in CMakeLists.txt and included files are all executed
>>>>>> during configure time, as the files are parsed and processed by CMake. Some
>>>>>> of these commands have immediate effects on the outside environment (e.g.
>>>>>> file(WRITE), execute_process(), try_compile()), some affect the CMake
>>>>>> processing control flow (e.g. include(), foreach()), some fill in or modify
>>>>>> CMake's data structures (e.g. set(), add_executable(), install(),
>>>>>> add_custom_command()).
>>>>>>
>>>>>> So there are no CMake commands executed after configure time. The
>>>>>> value specified in a COMMAND argument of add_custom_command() executes at
>>>>>> build time (driven by the build system), but the add_custom_command()
>>>>>> itself executes at CMake time.
>>>>>>
>>>>>> I hope I've clarified it a little bit.
>>>>>>
>>>>>> Petr
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jan 29, 2015 at 9:36 PM, Chris Johnson <cxjohnson at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Thanks; I appreciate the suggestions.
>>>>>>>
>>>>>>> One thing that jumped out is the thought that foreach() is executed
>>>>>>> at "configuration" time, not "run" time. To me, that was
>>>>>>> counter-intuitive, but makes more sense once more thoroughly considered.
>>>>>>>
>>>>>>> But I have to admit, I've never seen anything in the documentation
>>>>>>> which lists all of the commands which execute at configuration time, and
>>>>>>> which are executed at run time. Is there such documentation? A link to it
>>>>>>> would be great.
>>>>>>>
>>>>>>> On Thu, Jan 29, 2015 at 9:51 AM, Thompson, KT <kgt at lanl.gov> wrote:
>>>>>>>
>>>>>>>> Hi Chris,
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Why not use a foreach loop? Something like this…
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> add_executable(myprog myprog.cpp)
>>>>>>>>
>>>>>>>> set( files file001 fiile002 …)
>>>>>>>>
>>>>>>>> foreach( file in files)
>>>>>>>>
>>>>>>>> add_custom_command(OUTPUT ${file}.out
>>>>>>>>
>>>>>>>> COMMAND /bin/cp ${CMAKE_CURRENT_SOURCE_DIR}/${file}.in .
>>>>>>>>
>>>>>>>> COMMAND myprog ${file}.in ${file}.out
>>>>>>>>
>>>>>>>> DEPENDS ${file}.in myprog.cpp
>>>>>>>>
>>>>>>>> )
>>>>>>>>
>>>>>>>> endforeach()
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> For portability, you might want to use ‘${CMAKE_COMMAND} –E
>>>>>>>> copy_if_different’ in place of ‘/bin/cp’ and CMAKE_CURRENT_BINARY_DIR in
>>>>>>>> place of ‘.’
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> -kt
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> *From:* CMake [mailto:cmake-bounces at cmake.org] *On Behalf Of *Chris
>>>>>>>> Johnson
>>>>>>>> *Sent:* Thursday, January 29, 2015 8:09 AM
>>>>>>>> *To:* cmake at cmake.org
>>>>>>>> *Subject:* [CMake] Generated files?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> I am looking for suggestions on how to solve the following
>>>>>>>> problem. I need a solution which is easy to understand and safe from
>>>>>>>> future maintenance errors.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> There is a project directory with one C++ program and a large
>>>>>>>> number of "input" data files.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> The C++ program needs to be compiled, and then run against each of
>>>>>>>> the input files to generate an output file of the same base name but with a
>>>>>>>> different output suffix. E.g. from the command line it might be:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> % myprog file001.in file001.out
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Finally, the output files need to be installed in the proper
>>>>>>>> location.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> For one input file, it might look like this:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> add_executable(myprog myprog.cpp)
>>>>>>>>
>>>>>>>> add_custom_command(OUTPUT file001.out
>>>>>>>>
>>>>>>>> COMMAND /bin/cp ${CMAKE_CURRENT_SOURCE_DIR}/file001.in .
>>>>>>>>
>>>>>>>> COMMAND myprog file001.in file001.out
>>>>>>>>
>>>>>>>> DEPENDS files001.in myprog.cpp
>>>>>>>>
>>>>>>>> )
>>>>>>>>
>>>>>>>> install(FILES ${CMAKE_CURRENT_BINARY_DIR}/file001.out
>>>>>>>> DESTINATION share/docs)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Listing all of the input files in my CMakeLists.txt file is not a
>>>>>>>> problem, but it would be nice to avoid having to list the corresponding
>>>>>>>> output files.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> How can I build a CMakeLists.txt file which will succinctly handle
>>>>>>>> this situation?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> ..chris
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Powered by www.kitware.com
>>>>>>>
>>>>>>> Please keep messages on-topic and check the CMake FAQ at:
>>>>>>> http://www.cmake.org/Wiki/CMake_FAQ
>>>>>>>
>>>>>>> Kitware offers various services to support the CMake community. For
>>>>>>> more information on each offering, please visit:
>>>>>>>
>>>>>>> CMake Support: http://cmake.org/cmake/help/support.html
>>>>>>> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>>>>>>> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>>>>>>
>>>>>>> Visit other Kitware open-source projects at
>>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>>
>>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>>> http://public.kitware.com/mailman/listinfo/cmake
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cmake/attachments/20150130/ba172d45/attachment-0001.html>
More information about the CMake
mailing list