[CMake] Generated files?
Chris Johnson
cxjohnson at gmail.com
Fri Jan 30 12:07:29 EST 2015
I'm trying to use the method outlined in your #2, with the list of files.
But I get circular dependency errors, with messages like this one (for each
file):
make[2]: Circular validate/X12_Standards/997_5010.x12 <-
validate/X12_Standards/997_5010.x12 dependency dropped.
On Fri, Jan 30, 2015 at 10:28 AM, Petr Kmoch <petr.kmoch at gmail.com> wrote:
> I believe the classic "driving target" approach could be used here. Add a
> custom target which will drive the custom commands by depending on their
> outputs:
>
> add_custom_target(
> RunMyProg ALL
> DEPENDS
> ${CMAKE_CURRENT_BINARY_DIR}/file001.out
> ${CMAKE_CURRENT_BINARY_DIR}/file002.out
> #...
> COMMENT "Processing files with MyProg"
> )
>
> Notice the ALL after the target name - this makes the target part of 'make
> all' or equivalent.
>
> Custom targets are always considered out of date (always rebuilt), but
> this one has no COMMAND and so building it does nothing. It does make sure
> all its dependencies (the .out files) are up to date, though, and if they
> are not, it will run the rules to build them. These rules are the custom
> commands you've created earlier.
>
> This "custom target driving custom commands" is a very idiomatic pattern
> in CMake.
>
> Of course, you don't want to list the files by hand. You could modify my
> earlier function in one of two ways:
>
> 1. Have each invocation append to a list:
>
> set(OutputFileList "")
>
> function(ProcessFile inFile)
> get_filename_component(outFile ${inFile} NAME_WE)
> set(outFile ${inFile}.out)
> add_custom_command(
> OUTPUT ${outFile}
> COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/${inFile}
> ${CMAKE_CURRENT_BINARY_DIR}
> COMMAND myprog ${inFile} ${outFile}
> DEPENDS ${inFile} myprog.cpp
> )
> install(FILES ${CMAKE_CURRENT_BINARY_DIR}/${outFile} DESTINATION
> share/docs)
> set(OutputFileList ${OutputFileList}
> ${CMAKE_CURRENT_BINARY_DIR}/${outFile} PARENT_SCOPE) # This line is new
> endfunction()
>
> # process files
> ProcessFile(file001.in)
> ProcessFile(file002.in)
> ProcessFile(file003.in)
>
>
> # now use the list in the custom target:
> add_custom_target(
> RunMyProg ALL
> DEPENDS ${OutputFileList}
> COMMENT "Processing files with MyProg"
> )
>
>
> Or, 2., refactor the function to take a list of files and do everything
> itself:
>
> function(ProcessFiles)
> set(OutputFileList "")
> foreach(inFile IN LISTS ARGN)
> get_filename_component(outFile ${inFile} NAME_WE)
> set(outFile ${inFile}.out)
> add_custom_command(
> OUTPUT ${outFile}
> COMMAND ${CMAKE_COMMAND} -E copy
> ${CMAKE_CURRENT_SOURCE_DIR}/${inFile} ${CMAKE_CURRENT_BINARY_DIR}
> COMMAND myprog ${inFile} ${outFile}
> DEPENDS ${inFile} myprog.cpp
> )
> list(APPEND OutputFileList ${CMAKE_CURRENT_BINARY_DIR}/${outFile})
> endforeach()
> install(FILES ${OutputFileList} DESTINATION share/docs)
> add_custom_target(
> RunMyProg ALL
> DEPENDS ${OutputFileList}
> COMMENT "Processing files with MyProg"
> )
> endfunction()
>
> # now just call it once with the list of files:
> ProcessFiles(
> file001.in
> file002.in
> file003.in
> )
>
>
> You can naturally enlist file(GLOB) to grab the list of files, as J Decker
> suggests. The downside, as with any list of source files, is that CMake
> will not know that it needs to re-run when you simply add a new file which
> matches the glob. If you list the files explicitly, you have to edit the
> CMakeList to add the file, and CMake will know to re-run at next build.
>
> Petr
>
>
>
>
>
> On Fri, Jan 30, 2015 at 4:40 PM, Chris Johnson <cxjohnson at gmail.com>
> wrote:
>
>> Petr, thanks. That makes a lot of sense.
>>
>> Now I am struggling with causing all of these files to be generated when
>> I do the default build target, e.g. just "make" in the build directory.
>> That is, I want myprog to be compiled, and then all of the files processed
>> by it with just one build command. I assume that's possible, but have not
>> quite yet been able to make it happen.
>>
>> On Fri, Jan 30, 2015 at 9:11 AM, Petr Kmoch <petr.kmoch at gmail.com> wrote:
>>
>>> As you say, **make** is fairly powerful at this. Other build tools may
>>> not be, it might even be an alien concept to them. AFAIK, Ninja requires an
>>> explicit list of files & dependencies, for example. Visual Studio solutions
>>> can only contain explicitly listed files too.
>>>
>>> CMake projects are supposed to work with all generators which CMake
>>> supports (within reason). There are some generator-specific features in
>>> CMake, but the intent is to keep the set of these to a minimum.
>>>
>>> The list of CMake commands which directly create build-time "commands"
>>> (such as Makefile rules) is rather short:
>>>
>>> add_custom_command()
>>> add_custom_target()
>>> add_executable()
>>> add_library()
>>>
>>> Other commands modify the buildsystem (e.g. add_definitions(),
>>> target_link_libraries(), or setting properties such as COMPILE_OPTIONS),
>>> but they don't really add "commands" to it.
>>>
>>> Petr
>>>
>>>
>>> On Fri, Jan 30, 2015 at 3:27 PM, Chris Johnson <cxjohnson at gmail.com>
>>> wrote:
>>>
>>>> That does clarify a few things yes. Thank you.
>>>>
>>>> By "configure" time I had meant the first two items you listed as
>>>> Configure and Generate. By "run" time I had meant what you called Build.
>>>>
>>>> One place I am unclear on is exactly which commands in CMakeLists.txt
>>>> get translated into Build time commands in buildsystem files (in my case,
>>>> make). For example, make is fairly powerful at generating substitution
>>>> file names, e.g. creating prog.o from prog.cpp, or file1.out from
>>>> file1.in. Make is also capable of running full shell scripts within a
>>>> makefile. But there's no obvious way to drive any kind of automation from
>>>> CMake that make is capable of.
>>>>
>>>> As a programmer, my first thought when having to process a large list
>>>> of files is to do it through some sort of automation, rather than generate
>>>> a brute force solution. It appears that with CMake, the best one can hope
>>>> for is to automate the creation of the brute-force solution.
>>>>
>>>> --
>>>> Cheers,
>>>> ..chris
>>>>
>>>> On Fri, Jan 30, 2015 at 4:32 AM, Petr Kmoch <petr.kmoch at gmail.com>
>>>> wrote:
>>>>
>>>>> It's not clear to me what you mean with "run" time. There are the
>>>>> following "times" involved in a CMake-based project:
>>>>>
>>>>> * Configure time:
>>>>> CMake is parsing CMakeLists.txt files and files included from those
>>>>> and executing ALL CMake commands found in them. Variables expansion takes
>>>>> place. Data structures in CMake's memory are filled.
>>>>>
>>>>> * Generate time:
>>>>> CMake processes the data structures in its memory to write buildsystem
>>>>> files (Makefiles, solutions etc.) to disk. Generator expressions are
>>>>> expanded. Files are generated from data specified in file(GENERATE ...)
>>>>> commands. cmake_install.cmake file is written based on the contents of data
>>>>> structures filled by install() commands.
>>>>>
>>>>> * Build time:
>>>>> CMake is not running (except for driving the build if you do that with
>>>>> `cmake --build`). The build tool (make, ninja, devenv, msbuild, ...) is
>>>>> running, processing the buildsystem files and generating compiled binaries.
>>>>> At this time, commands which were set up by add_custom_command() and
>>>>> add_custom_target() are executed as driven by the build tool.
>>>>>
>>>>> * Install time (optional):
>>>>> Running `make install` or similarly triggering the install process
>>>>> causes CMake to run in installation mode. This CMake reads the
>>>>> cmake_install.cmake file and executes the commands there. These are mainly
>>>>> file() commands which copy files to the desired location.
>>>>>
>>>>> CMake commands in CMakeLists.txt and included files are all executed
>>>>> during configure time, as the files are parsed and processed by CMake. Some
>>>>> of these commands have immediate effects on the outside environment (e.g.
>>>>> file(WRITE), execute_process(), try_compile()), some affect the CMake
>>>>> processing control flow (e.g. include(), foreach()), some fill in or modify
>>>>> CMake's data structures (e.g. set(), add_executable(), install(),
>>>>> add_custom_command()).
>>>>>
>>>>> So there are no CMake commands executed after configure time. The
>>>>> value specified in a COMMAND argument of add_custom_command() executes at
>>>>> build time (driven by the build system), but the add_custom_command()
>>>>> itself executes at CMake time.
>>>>>
>>>>> I hope I've clarified it a little bit.
>>>>>
>>>>> Petr
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Jan 29, 2015 at 9:36 PM, Chris Johnson <cxjohnson at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Thanks; I appreciate the suggestions.
>>>>>>
>>>>>> One thing that jumped out is the thought that foreach() is executed
>>>>>> at "configuration" time, not "run" time. To me, that was
>>>>>> counter-intuitive, but makes more sense once more thoroughly considered.
>>>>>>
>>>>>> But I have to admit, I've never seen anything in the documentation
>>>>>> which lists all of the commands which execute at configuration time, and
>>>>>> which are executed at run time. Is there such documentation? A link to it
>>>>>> would be great.
>>>>>>
>>>>>> On Thu, Jan 29, 2015 at 9:51 AM, Thompson, KT <kgt at lanl.gov> wrote:
>>>>>>
>>>>>>> Hi Chris,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Why not use a foreach loop? Something like this…
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> add_executable(myprog myprog.cpp)
>>>>>>>
>>>>>>> set( files file001 fiile002 …)
>>>>>>>
>>>>>>> foreach( file in files)
>>>>>>>
>>>>>>> add_custom_command(OUTPUT ${file}.out
>>>>>>>
>>>>>>> COMMAND /bin/cp ${CMAKE_CURRENT_SOURCE_DIR}/${file}.in .
>>>>>>>
>>>>>>> COMMAND myprog ${file}.in ${file}.out
>>>>>>>
>>>>>>> DEPENDS ${file}.in myprog.cpp
>>>>>>>
>>>>>>> )
>>>>>>>
>>>>>>> endforeach()
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> For portability, you might want to use ‘${CMAKE_COMMAND} –E
>>>>>>> copy_if_different’ in place of ‘/bin/cp’ and CMAKE_CURRENT_BINARY_DIR in
>>>>>>> place of ‘.’
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> -kt
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *From:* CMake [mailto:cmake-bounces at cmake.org] *On Behalf Of *Chris
>>>>>>> Johnson
>>>>>>> *Sent:* Thursday, January 29, 2015 8:09 AM
>>>>>>> *To:* cmake at cmake.org
>>>>>>> *Subject:* [CMake] Generated files?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I am looking for suggestions on how to solve the following
>>>>>>> problem. I need a solution which is easy to understand and safe from
>>>>>>> future maintenance errors.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> There is a project directory with one C++ program and a large number
>>>>>>> of "input" data files.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> The C++ program needs to be compiled, and then run against each of
>>>>>>> the input files to generate an output file of the same base name but with a
>>>>>>> different output suffix. E.g. from the command line it might be:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> % myprog file001.in file001.out
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Finally, the output files need to be installed in the proper
>>>>>>> location.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> For one input file, it might look like this:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> add_executable(myprog myprog.cpp)
>>>>>>>
>>>>>>> add_custom_command(OUTPUT file001.out
>>>>>>>
>>>>>>> COMMAND /bin/cp ${CMAKE_CURRENT_SOURCE_DIR}/file001.in .
>>>>>>>
>>>>>>> COMMAND myprog file001.in file001.out
>>>>>>>
>>>>>>> DEPENDS files001.in myprog.cpp
>>>>>>>
>>>>>>> )
>>>>>>>
>>>>>>> install(FILES ${CMAKE_CURRENT_BINARY_DIR}/file001.out
>>>>>>> DESTINATION share/docs)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Listing all of the input files in my CMakeLists.txt file is not a
>>>>>>> problem, but it would be nice to avoid having to list the corresponding
>>>>>>> output files.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> How can I build a CMakeLists.txt file which will succinctly handle
>>>>>>> this situation?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> ..chris
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Powered by www.kitware.com
>>>>>>
>>>>>> Please keep messages on-topic and check the CMake FAQ at:
>>>>>> http://www.cmake.org/Wiki/CMake_FAQ
>>>>>>
>>>>>> Kitware offers various services to support the CMake community. For
>>>>>> more information on each offering, please visit:
>>>>>>
>>>>>> CMake Support: http://cmake.org/cmake/help/support.html
>>>>>> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>>>>>> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>>>>>
>>>>>> Visit other Kitware open-source projects at
>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>
>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>> http://public.kitware.com/mailman/listinfo/cmake
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cmake/attachments/20150130/9ecba605/attachment-0001.html>
More information about the CMake
mailing list