<div dir="ltr"><div class="gmail_default" style="font-family:'courier new',monospace">I'm trying to use the method outlined in your #2, with the list of files. But I get circular dependency errors, with messages like this one (for each file):</div><div class="gmail_default" style="font-family:'courier new',monospace"><br></div><div class="gmail_default" style="font-family:'courier new',monospace">make[2]: Circular validate/X12_Standards/997_5010.x12 <- validate/X12_Standards/997_5010.x12 dependency dropped.<br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jan 30, 2015 at 10:28 AM, Petr Kmoch <span dir="ltr"><<a href="mailto:petr.kmoch@gmail.com" target="_blank">petr.kmoch@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div><div><div>I believe the classic "driving target" approach could be used here. Add a custom target which will drive the custom commands by depending on their outputs:<br><br></div>add_custom_target(<br></div> RunMyProg ALL<br></div> DEPENDS<br> ${CMAKE_CURRENT_BINARY_DIR}/file001.out<br> ${CMAKE_CURRENT_BINARY_DIR}/file002.out<br> #...<br></div> COMMENT "Processing files with MyProg"<br>)<br><br></div>Notice the ALL after the target name - this makes the target part of 'make all' or equivalent.<br><br>Custom targets are always considered out of date (always rebuilt), but this one has no COMMAND and so building it does nothing. It does make sure all its dependencies (the .out files) are up to date, though, and if they are not, it will run the rules to build them. These rules are the custom commands you've created earlier.<br><br>This "custom target driving custom commands" is a very idiomatic pattern in CMake.<br><br></div>Of course, you don't want to list the files by hand. You could modify my earlier function in one of two ways:<br><br></div>1. Have each invocation append to a list:<br><br>set(OutputFileList "")<span class=""><br><br>function(ProcessFile inFile)<br><div> get_filename_component(outFile ${inFile} NAME_WE)<br></div></span><div> set(outFile ${inFile}.out)<br></div><span class=""> add_custom_command(<br> OUTPUT ${outFile}<br> COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/${inFile} ${CMAKE_CURRENT_BINARY_DIR}<br> COMMAND myprog ${inFile} ${outFile}<br></span> DEPENDS ${inFile} myprog.cpp<br> )<br> install(FILES ${CMAKE_CURRENT_BINARY_DIR}/${outFile} DESTINATION share/docs)<br></div> set(OutputFileList ${OutputFileList} ${CMAKE_CURRENT_BINARY_DIR}/${outFile} PARENT_SCOPE) # This line is new<br><div><div dir="ltr"><div><div><div><div><div>endfunction()<br><br></div><div># process files<br></div><div>ProcessFile(<a href="http://file001.in" target="_blank">file001.in</a>)<br>ProcessFile(<a href="http://file002.in" target="_blank">file002.in</a>)<br>ProcessFile(<a href="http://file003.in" target="_blank">file003.in</a>)<br><br></div><div><br></div><div># now use the list in the custom target:<br>add_custom_target(<br></div><div> RunMyProg ALL<br></div><div> DEPENDS ${OutputFileList}<br></div><div> COMMENT "Processing files with MyProg"</div><div>)<br><br><br></div><div>Or, 2., refactor the function to take a list of files and do everything itself:<br></div><div><br>function(ProcessFiles)<br></div><div> set(OutputFileList "")<br></div><div> foreach(inFile IN LISTS ARGN)<br></div><div><div> get_filename_component(outFile ${inFile} NAME_WE)<br></div><div> set(outFile ${inFile}.out)<br></div><span class=""> add_custom_command(<br> OUTPUT ${outFile}<br> COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/${inFile} ${CMAKE_CURRENT_BINARY_DIR}<br> COMMAND myprog ${inFile} ${outFile}<br> DEPENDS ${inFile} myprog.cpp<br> )<br></span></div><div> list(APPEND OutputFileList ${CMAKE_CURRENT_BINARY_DIR}/${outFile})<br></div><div> endforeach()<br></div><div> install(FILES ${OutputFileList} DESTINATION share/docs)<br></div><div> add_custom_target(<br><div> RunMyProg ALL<br></div><div> DEPENDS ${OutputFileList}<br></div><div> COMMENT "Processing files with MyProg"</div> )<br></div><div>endfunction()<br><br></div><div># now just call it once with the list of files:<br></div><div>ProcessFiles(<br> <a href="http://file001.in" target="_blank">file001.in</a><br> <a href="http://file002.in" target="_blank">file002.in</a><br></div><div> <a href="http://file003.in" target="_blank">file003.in</a><br>)<br></div><div> </div><div><br></div><div>You can naturally enlist file(GLOB) to grab the list of files, as J Decker suggests. The downside, as with any list of source files, is that CMake will not know that it needs to re-run when you simply add a new file which matches the glob. If you list the files explicitly, you have to edit the CMakeList to add the file, and CMake will know to re-run at next build.<span class="HOEnZb"><font color="#888888"><br><br>Petr<br></font></span></div><div><br></div></div></div></div></div></div><br><div><div><div><div><div><div><div><div><br><br></div></div></div></div></div></div></div></div></div><div><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jan 30, 2015 at 4:40 PM, Chris Johnson <span dir="ltr"><<a href="mailto:cxjohnson@gmail.com" target="_blank">cxjohnson@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:courier new,monospace">Petr, thanks. That makes a lot of sense.</div><div style="font-family:courier new,monospace"><br></div><div style="font-family:courier new,monospace">Now I am struggling with causing all of these files to be generated when I do the default build target, e.g. just "make" in the build directory. That is, I want myprog to be compiled, and then all of the files processed by it with just one build command. I assume that's possible, but have not quite yet been able to make it happen.</div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jan 30, 2015 at 9:11 AM, Petr Kmoch <span dir="ltr"><<a href="mailto:petr.kmoch@gmail.com" target="_blank">petr.kmoch@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div>As you say, **make** is fairly powerful at this. Other build tools may not be, it might even be an alien concept to them. AFAIK, Ninja requires an explicit list of files & dependencies, for example. Visual Studio solutions can only contain explicitly listed files too.<br><br>CMake projects are supposed to work with all generators which CMake supports (within reason). There are some generator-specific features in CMake, but the intent is to keep the set of these to a minimum.<br><br></div>The list of CMake commands which directly create build-time "commands" (such as Makefile rules) is rather short:<br><br><div>add_custom_command()<br>add_custom_target()<br></div>add_executable()<br>add_library()<br><br></div><div>Other commands modify the buildsystem (e.g. add_definitions(), target_link_libraries(), or setting properties such as COMPILE_OPTIONS), but they don't really add "commands" to it.<span><font color="#888888"><br><br></font></span></div><span><font color="#888888"><div>Petr<br></div><br></font></span></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jan 30, 2015 at 3:27 PM, Chris Johnson <span dir="ltr"><<a href="mailto:cxjohnson@gmail.com" target="_blank">cxjohnson@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:courier new,monospace">That does clarify a few things yes. Thank you.</div><div style="font-family:courier new,monospace"><br></div><div style="font-family:courier new,monospace">By "configure" time I had meant the first two items you listed as Configure and Generate. By "run" time I had meant what you called Build. </div><div style="font-family:courier new,monospace"><br></div><div style="font-family:courier new,monospace">One place I am unclear on is exactly which commands in CMakeLists.txt get translated into Build time commands in buildsystem files (in my case, make). For example, make is fairly powerful at generating substitution file names, e.g. creating prog.o from prog.cpp, or file1.out from <a href="http://file1.in" target="_blank">file1.in</a>. Make is also capable of running full shell scripts within a makefile. But there's no obvious way to drive any kind of automation from CMake that make is capable of.</div><div style="font-family:courier new,monospace"><br></div><div style="font-family:courier new,monospace">As a programmer, my first thought when having to process a large list of files is to do it through some sort of automation, rather than generate a brute force solution. It appears that with CMake, the best one can hope for is to automate the creation of the brute-force solution.</div><div style="font-family:courier new,monospace"><br></div><div style="font-family:courier new,monospace">--</div><div style="font-family:courier new,monospace">Cheers,</div><div style="font-family:courier new,monospace">..chris</div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jan 30, 2015 at 4:32 AM, Petr Kmoch <span dir="ltr"><<a href="mailto:petr.kmoch@gmail.com" target="_blank">petr.kmoch@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div><div>It's not clear to me what you mean with "run" time. There are the following "times" involved in a CMake-based project:<br><br></div>* Configure time:<br>CMake is parsing CMakeLists.txt files and files included from those and executing ALL CMake commands found in them. Variables expansion takes place. Data structures in CMake's memory are filled.<br></div><br>* Generate time:<br>CMake processes the data structures in its memory to write buildsystem files (Makefiles, solutions etc.) to disk. Generator expressions are expanded. Files are generated from data specified in file(GENERATE ...) commands. cmake_install.cmake file is written based on the contents of data structures filled by install() commands.<br><br></div>* Build time:<br></div>CMake is not running (except for driving the build if you do that with `cmake --build`). The build tool (make, ninja, devenv, msbuild, ...) is running, processing the buildsystem files and generating compiled binaries. At this time, commands which were set up by add_custom_command() and add_custom_target() are executed as driven by the build tool.<br><br></div>* Install time (optional):<br></div>Running `make install` or similarly triggering the install process causes CMake to run in installation mode. This CMake reads the cmake_install.cmake file and executes the commands there. These are mainly file() commands which copy files to the desired location.<br><br></div><div>CMake commands in CMakeLists.txt and included files are all executed during configure time, as the files are parsed and processed by CMake. Some of these commands have immediate effects on the outside environment (e.g. file(WRITE), execute_process(), try_compile()), some affect the CMake processing control flow (e.g. include(), foreach()), some fill in or modify CMake's data structures (e.g. set(), add_executable(), install(), add_custom_command()).<br><br></div><div>So there are no CMake commands executed after configure time. The value specified in a COMMAND argument of add_custom_command() executes at build time (driven by the build system), but the add_custom_command() itself executes at CMake time.<br></div><br><div>I hope I've clarified it a little bit.<span><font color="#888888"><br><br>Petr<br></font></span></div><div><div><div><div><div><div><br><br></div></div></div></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div>On Thu, Jan 29, 2015 at 9:36 PM, Chris Johnson <span dir="ltr"><<a href="mailto:cxjohnson@gmail.com" target="_blank">cxjohnson@gmail.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div><div dir="ltr"><div style="font-family:courier new,monospace">Thanks; I appreciate the suggestions.</div><div style="font-family:courier new,monospace"><br></div><div style="font-family:courier new,monospace">One thing that jumped out is the thought that foreach() is executed at "configuration" time, not "run" time. To me, that was counter-intuitive, but makes more sense once more thoroughly considered.</div><div style="font-family:courier new,monospace"><br></div><div style="font-family:courier new,monospace">But I have to admit, I've never seen anything in the documentation which lists all of the commands which execute at configuration time, and which are executed at run time. Is there such documentation? A link to it would be great.</div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 29, 2015 at 9:51 AM, Thompson, KT <span dir="ltr"><<a href="mailto:kgt@lanl.gov" target="_blank">kgt@lanl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div link="blue" vlink="purple" lang="EN-US">
<div>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)">Hi Chris,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)">Why not use a foreach loop? Something like this…<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)">add_executable(myprog myprog.cpp)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)">set( files file001 fiile002 …)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)">foreach( file in files)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)"> add_custom_command(OUTPUT ${file}.out<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)"> COMMAND /bin/cp ${CMAKE_CURRENT_SOURCE_DIR}/${file}.in .<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)"> COMMAND myprog ${file}.in ${file}.out<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)"> DEPENDS ${file}.in myprog.cpp<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)"> )<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)">endforeach()<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)">For portability, you might want to use ‘${CMAKE_COMMAND} –E copy_if_different’ in place of ‘/bin/cp’ and CMAKE_CURRENT_BINARY_DIR in place of ‘.’<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)">-kt<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11pt;font-family:"Calibri",sans-serif"> CMake [mailto:<a href="mailto:cmake-bounces@cmake.org" target="_blank">cmake-bounces@cmake.org</a>]
<b>On Behalf Of </b>Chris Johnson<br>
<b>Sent:</b> Thursday, January 29, 2015 8:09 AM<br>
<b>To:</b> <a href="mailto:cmake@cmake.org" target="_blank">cmake@cmake.org</a><br>
<b>Subject:</b> [CMake] Generated files?<u></u><u></u></span></p><div><div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Cambria Math",serif"></span><span style="font-family:"Courier New"">I am looking for suggestions on how to solve the following problem. I need a solution which is easy to understand and safe from future maintenance
errors.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">There is a project directory with one C++ program and a large number of "input" data files. <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">The C++ program needs to be compiled, and then run against each of the input files to generate an output file of the same base name but with a different output suffix. E.g. from the command line
it might be:<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""> % myprog <a href="http://file001.in" target="_blank">file001.in</a> file001.out<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Finally, the output files need to be installed in the proper location.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">For one input file, it might look like this:<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""> add_executable(myprog myprog.cpp)<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""> add_custom_command(OUTPUT file001.out<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""> COMMAND /bin/cp ${CMAKE_CURRENT_SOURCE_DIR}/<a href="http://file001.in" target="_blank">file001.in</a> .<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""> COMMAND myprog
<a href="http://file001.in" target="_blank">file001.in</a> file001.out<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""> DEPENDS <a href="http://files001.in" target="_blank">
files001.in</a> myprog.cpp<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""> )<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""> install(FILES ${CMAKE_CURRENT_BINARY_DIR}/file001.out DESTINATION share/docs)<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Listing all of the input files in my CMakeLists.txt file is not a problem, but it would be nice to avoid having to list the corresponding output files.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">How can I build a CMakeLists.txt file which will succinctly handle this situation?<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">--<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Thanks,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">..chris</span><span style="font-family:"Cambria Math",serif"></span><span style="font-family:"Courier New""><u></u><u></u></span></p>
</div>
</div>
</div></div></div>
</div>
</blockquote></div><br></div>
</div></div><br></div></div><span>--<br>
<br>
Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
<br>
Please keep messages on-topic and check the CMake FAQ at: <a href="http://www.cmake.org/Wiki/CMake_FAQ" target="_blank">http://www.cmake.org/Wiki/CMake_FAQ</a><br>
<br>
Kitware offers various services to support the CMake community. For more information on each offering, please visit:<br>
<br>
CMake Support: <a href="http://cmake.org/cmake/help/support.html" target="_blank">http://cmake.org/cmake/help/support.html</a><br>
CMake Consulting: <a href="http://cmake.org/cmake/help/consulting.html" target="_blank">http://cmake.org/cmake/help/consulting.html</a><br>
CMake Training Courses: <a href="http://cmake.org/cmake/help/training.html" target="_blank">http://cmake.org/cmake/help/training.html</a><br>
<br>
Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="http://public.kitware.com/mailman/listinfo/cmake" target="_blank">http://public.kitware.com/mailman/listinfo/cmake</a><br></span></blockquote></div><br></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div></div></div></div>
</blockquote></div><br></div>