[CMake] avoid rebuilding targets depending on generated source files
Ajay Panyala
ajay at csc.lsu.edu
Sun Mar 4 22:53:26 EST 2012
So, my question again: What exactly does not work?
I.e., does "cmake -E copy_if_different"
- copy a file although the destination exists
and has the same content as the source, or
Sorry for the confusion.
I thought something like the above mentioned happened, but in reality
nothing like it happened. I must have done something
wrong by overlooking some detail. It works now.
*I did not initially run the CMakeLists you had provided.*
I had run my own version of CMakeLists by inserting some rules
from the CMakeLists file you had provided and I did something
wrong there. Hence, that did not work.
> It works for my CMakeLists as well now. What I did now is
>
> if(${GrammarSource}/test.g IS_NEWER_THAN ${PROJECT_BINARY_DIR}/test.g)
> ADD_CUSTOM_TARGET(...)
> ADD_CUSTOM_COMMAND(...)
>
> i.e I am totally avoiding the process of rebuilding the test.g file and
> copying the generated files to build/parser
> *IF* test.g has not been modified.
...and if test.g is once modified, you won't have a custom target which
regenerates the source files and copies them to the parser directory -
I'm pretty sure this is not what you intend. Recall the difference
between configuration time and build time in CMakified projects.
No, if test.g is modified the *if* condition is true and the new version
is copied over to the build directory; the custom target builds it
and the custom command moves the generated files into build/parser.
For the initial run of make test.g is not present in the build folder,
since it
was never copied before. For this case, I have another IF(EXISTS
build/test.g) conditional
which copies it from src to build folder and then executes it.
Thanks
Ajay
On Sun, Mar 4, 2012 at 7:35 PM, Michael Hertling <mhertling at online.de>wrote:
> On 03/05/2012 01:59 AM, Ajay Panyala wrote:
> > I use cmake version 2.8.3.
> >
> > If I use CMakeLists from your previous reply, it avoids overwriting files
> > when X=0.
> > I have attached the output of your script.
>
> Actually, this is exactly what I was talking about: The second "make
> X=0" invocation rewrites the source files - i.e., they receive a new
> timestamp - but their contents do not change. Thus, the destination
> files in the parser directory are *not* touched by the second "make
> X=0", and this can be seen clearly by reference to the timestamps.
> It's the subsequent "make X=1" which provides for a new content of
> the source files and makes "cmake -E copy_if_different" copy the
> sources to the destinations. That's why I said everything works
> as expected, but you wrote:
>
> >>> It does not work. The files are still overwritten.
>
> So, my question again: What exactly does not work?
>
> I.e., does "cmake -E copy_if_different"
>
> - copy a file although the destination exists
> and has the same content as the source, or
>
> - not copy a file although the destination does not
> exist or does not have the same content as the source?
>
> > It works for my CMakeLists as well now. What I did now is
> >
> > if(${GrammarSource}/test.g IS_NEWER_THAN ${PROJECT_BINARY_DIR}/test.g)
> > ADD_CUSTOM_TARGET(...)
> > ADD_CUSTOM_COMMAND(...)
> >
> > i.e I am totally avoiding the process of rebuilding the test.g file and
> > copying the generated files to build/parser
> > *IF* test.g has not been modified.
>
> ...and if test.g is once modified, you won't have a custom target which
> regenerates the source files and copies them to the parser directory -
> I'm pretty sure this is not what you intend. Recall the difference
> between configuration time and build time in CMakified projects.
>
> > PS: Could you set up your e-mail client so that it marks quoted
> > text in some way? Currently, one can't distinguish between
> > the lines you are writing and the lines written by others.
> >
> > Sorry. I did not realize that. I use gmail and when I checked my sent
> > emails,
> > quoted text appears to be clearly marked. Is the quoted text in this
> email
> > marked properly.
>
> Yes, it is.
>
> Regards,
>
> Michael
>
> > On Sun, Mar 4, 2012 at 4:05 PM, Michael Hertling <mhertling at online.de
> >wrote:
> >
> >> On 03/04/2012 08:02 PM, Ajay Panyala wrote:
> >>> The following project is a boiled-down version of yours but doesn't
> >>> need any programs except for CMake - that's what I actually meant
> >>> with "minimal but complete" as I don't have the org.antlr.Tool
> >>> Java program:
> >>>
> >>> I am sorry. I was mainly trying to cleanup the big CMakeLists file
> >>> I had and removed about 10 different targets - all of which were
> >>> rebuilt because the 5 files (test*.*) were overwritten.
> >>>
> >>> If you want to try running the test project on your system
> >>>
> >>> test.g is at http://pastie.org/private/agzor3ibzoa5pom6q31qq
> >>>
> >>> org.antlr.Tool is at www.antlr.org/download/antlr-3.4-complete.jar
> >>>
> >>> After configuration, you can run the target by "make X=0" and check the
> >>> timestamps by "ls -l --full-time test.tokens parser src". Issuing "make
> >>> X=0" again reveals that the copied files aren't rewritten as it happens
> >>> after "make X=1". Thus, AFAICS, everything works as expected. Could you
> >>> check if the above-noted example also works on your system?
> >>>
> >>> It does not work. The files are still overwritten.
> >>
> >> Could you run the following shell script in a build tree configured
> >> with the CMakeLists.txt from my previous reply and post the output?
> >>
> >> # check.sh:
> >> make X=0 2>&1 > /dev/null
> >> echo "After make X=0 (1):"
> >> echo "-------------------"
> >> echo ""
> >> ls --full-time test.tokens src parser
> >> echo -ne "\n\n"
> >> sleep 3
> >> make X=0 2>&1 > /dev/null
> >> echo "After make X=0 (2):"
> >> echo "-------------------"
> >> echo ""
> >> ls --full-time test.tokens src parser
> >> echo -ne "\n\n"
> >> sleep 3
> >> make X=1 2>&1 > /dev/null
> >> echo "After make X=1:"
> >> echo "---------------"
> >> echo ""
> >> ls --full-time test.tokens src parser
> >> # End of check.sh
> >>
> >> BTW, which CMake version do you use?
> >>
> >> Regards,
> >>
> >> Michael
> >>
> >> PS: Could you set up your e-mail client so that it marks quoted
> >> text in some way? Currently, one can't distinguish between
> >> the lines you are writing and the lines written by others.
> >>
> >>> What exactly does not work with your example? You wrote:
> >>>
> >>>>>> I have 4 cmake -E copy_if_different commands, one for each file.
> >>>>>> Only the last file is not copied (if similar). [...]
> >>>
> >>> Does this mean that the last file out of four - in fact, your example
> >>> handles five files - is not copied *although* the source file and the
> >>> destination file are different, i.e. similar but not equal?
> >>>
> >>> Yes the file test.tokens is not copied overwritten since they are
> >>> exactly (diff) similar files. This is the case with the other 4 files
> as
> >>> well,
> >>> but they are still copied over and rewritten.
> >>>
> >>>
> >>> You wrote further:
> >>>
> >>>>>> [...] The others are copied
> >>>>>> even if they are the same.
> >>>>>>
> >>>>>> I verfied that they are the same with a diff.
> >>>
> >>> Does this mean that source files are copied *although* they are equal
> >>> to their respective destination file? How do you determine that they
> >>> have been copied? Do you check the timestamps? With --full-time?
> >>>
> >>> Yes, I do check with ls -l --full-time. Except test.tokens, all the
> other
> >>> files
> >>> are copied over (rewritten) even though they are exactly the same (diff
> >>> same I mean).
> >>>
> >>> This is what is confusing me about the behavior of copy_if_different.
> >>> that is why it works only with test.tokens and not others.
> >>>
> >>> PS: Does org.antlr.Tool write to the source tree? If so: Don't do that.
> >>>
> >>> Yes, it generates the files in the source dir itself (where test.g is
> >>> present)
> >>> I now modified CMakeLists to copy test.g to the project build folder
> and
> >>> run it there. The new CMakeLists is at
> >>>
> >>> http://pastie.org/private/p1yi0l8so9cqimqlywfmhw
> >>>
> >>>
> >>> Thank You
> >>> Ajay
> >>>
> >>> On Sun, Mar 4, 2012 at 12:52 AM, Michael Hertling <mhertling at online.de
> >>> wrote:
> >>>
> >>>> On 03/04/2012 01:06 AM, Ajay Panyala wrote:
> >>>>> Please provide a minimal but complete example for this issue.
> >>>>>
> >>>>> Please find it in the following link
> >>>>> http://pastie.org/private/pd13u33s9xpfihf2dbzc1q
> >>>>
> >>>>
> >>>
> >>>
> >>>> The following project is a boiled-down version of yours but doesn't
> >>>> need any programs except for CMake - that's what I actually meant
> >>>> with "minimal but complete" as I don't have the org.antlr.Tool
> >>>> Java program:
> >>>>
> >>>> CMAKE_MINIMUM_REQUIRED(VERSION 2.8 FATAL_ERROR)
> >>>> PROJECT(P NONE)
> >>>> SET(CMAKE_VERBOSE_MAKEFILE ON)
> >>>>
> >>>> SET(GrammarSource ${PROJECT_BINARY_DIR}/src)
> >>>> FILE(MAKE_DIRECTORY ${PROJECT_BINARY_DIR}/src)
> >>>>
> >>>> ADD_CUSTOM_TARGET(GrammarBuild ALL
> >>>> COMMAND ${CMAKE_COMMAND} -E echo "\${X}"
> >>>> > ${GrammarSource}/testLexer.h
> >>>> COMMAND ${CMAKE_COMMAND} -E echo "\${X}"
> >>>> > ${GrammarSource}/testLexer.c
> >>>> COMMAND ${CMAKE_COMMAND} -E echo "\${X}"
> >>>> > ${GrammarSource}/testParser.h
> >>>> COMMAND ${CMAKE_COMMAND} -E echo "\${X}"
> >>>> > ${GrammarSource}/testParser.c
> >>>> COMMAND ${CMAKE_COMMAND} -E echo "\${X}"
> >>>> > ${PROJECT_BINARY_DIR}/test.tokens
> >>>> )
> >>>>
> >>>> ADD_CUSTOM_COMMAND(TARGET GrammarBuild POST_BUILD
> >>>> COMMAND ${CMAKE_COMMAND} -E copy_if_different
> >>>> ${GrammarSource}/testLexer.h
> >>>> ${PROJECT_BINARY_DIR}/parser/testLexer.h
> >>>> COMMAND ${CMAKE_COMMAND} -E copy_if_different
> >>>> ${GrammarSource}/testLexer.c
> >>>> ${PROJECT_BINARY_DIR}/parser/testLexer.c
> >>>> COMMAND ${CMAKE_COMMAND} -E copy_if_different
> >>>> ${GrammarSource}/testParser.h
> >>>> ${PROJECT_BINARY_DIR}/parser/testParser.h
> >>>> COMMAND ${CMAKE_COMMAND} -E copy_if_different
> >>>> ${GrammarSource}/testParser.c
> >>>> ${PROJECT_BINARY_DIR}/parser/testParser.c
> >>>> COMMAND ${CMAKE_COMMAND} -E copy_if_different
> >>>> ${PROJECT_BINARY_DIR}/test.tokens
> >>>> ${PROJECT_BINARY_DIR}/parser/test.tokens
> >>>> )
> >>>>
> >>>> After configuration, you can run the target by "make X=0" and check
> the
> >>>> timestamps by "ls -l --full-time test.tokens parser src". Issuing
> "make
> >>>> X=0" again reveals that the copied files aren't rewritten as it
> happens
> >>>> after "make X=1". Thus, AFAICS, everything works as expected. Could
> you
> >>>> check if the above-noted example also works on your system?
> >>>>
> >>>> What exactly does not work with your example? You wrote:
> >>>>
> >>>>>>> I have 4 cmake -E copy_if_different commands, one for each file.
> >>>>>>> Only the last file is not copied (if similar). [...]
> >>>>
> >>>> Does this mean that the last file out of four - in fact, your example
> >>>> handles five files - is not copied *although* the source file and the
> >>>> destination file are different, i.e. similar but not equal?
> >>>>
> >>>> You wrote further:
> >>>>
> >>>>>>> [...] The others are copied
> >>>>>>> even if they are the same.
> >>>>>>>
> >>>>>>> I verfied that they are the same with a diff.
> >>>>
> >>>> Does this mean that source files are copied *although* they are equal
> >>>> to their respective destination file? How do you determine that they
> >>>> have been copied? Do you check the timestamps? With --full-time?
> >>>>
> >>>> Regards,
> >>>>
> >>>> Michael
> >>>>
> >>>> PS: Does org.antlr.Tool write to the source tree? If so: Don't do
> that.
> >>>>
> >>>>> On Sat, Mar 3, 2012 at 3:54 PM, Michael Hertling <
> mhertling at online.de
> >>>>> wrote:
> >>>>>
> >>>>>> On 03/04/2012 12:14 AM, Ajay Panyala wrote:
> >>>>>>> No, it wouldn't; check it out:
> >>>>>>>
> >>>>>>> % touch a
> >>>>>>> % rm -f b
> >>>>>>> % ls b
> >>>>>>> ls: cannot access b: No such file or directory
> >>>>>>> % cmake -E copy_if_different a b
> >>>>>>> % ls b
> >>>>>>> b
> >>>>>>> % cksum a b
> >>>>>>> 4294967295 0 a
> >>>>>>> 4294967295 0 b
> >>>>>>>
> >>>>>>> It works with one file, but I have 4 files that are generated.
> >>>>>>> I have 4 cmake -E copy_if_different commands, one for each file.
> >>>>>>> Only the last file is not copied (if similar). The others are
> copied
> >>>>>>> even if they are the same.
> >>>>>>>
> >>>>>>> I verfied that they are the same with a diff.
> >>>>>>>
> >>>>>>> Any idea what might be happening here ?
> >>>>>>
> >>>>>> Please provide a minimal but complete example for this issue.
> >>>>>>
> >>>>>> Regards,
> >>>>>>
> >>>>>> Michael
> >>>>>>
> >>>>>>> On Sat, Mar 3, 2012 at 2:47 PM, Michael Hertling <
> >> mhertling at online.de
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> On 03/03/2012 10:36 PM, Ajay Panyala wrote:
> >>>>>>>>> Try "cmake -E copy_if_different ..."
> >>>>>>>>>
> >>>>>>>>> cmake -E copy_if_different build/test1.c build/tests/test1.c
> >>>>>>>>>
> >>>>>>>>> That would work when make is run atleast once.
> >>>>>>>>> When running make for the 1st time test1.c was never
> >>>>>>>>> copied to build/tests before. So I would be comparing a file with
> >>>>>>>>> another non-existant file and that would result in an error
> halting
> >>>>>>>>> the make process.
> >>>>>>>>
> >>>>>>>> No, it wouldn't; check it out:
> >>>>>>>>
> >>>>>>>> % touch a
> >>>>>>>> % rm -f b
> >>>>>>>> % ls b
> >>>>>>>> ls: cannot access b: No such file or directory
> >>>>>>>> % cmake -E copy_if_different a b
> >>>>>>>> % ls b
> >>>>>>>> b
> >>>>>>>> % cksum a b
> >>>>>>>> 4294967295 0 a
> >>>>>>>> 4294967295 0 b
> >>>>>>>>
> >>>>>>>> Regards,
> >>>>>>>>
> >>>>>>>> Michael
> >>>>>>>>
> >>>>>>>>> On Sat, Mar 3, 2012 at 1:20 PM, Hendrik Sattler <
> >>>>>> post at hendrik-sattler.de
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Am Samstag, 3. März 2012, 21:41:49 schrieb Ajay Panyala:
> >>>>>>>>>>> I have a custom target which runs a command to generate
> >>>>>>>>>>> a C source file say test1.c
> >>>>>>>>>>>
> >>>>>>>>>>> ADD_CUSTOM_TARGET(TestGen ALL
> >>>>>>>>>>> COMMAND genExec ${PROJECT_SOURCE_DIR}/Main.java
> >>>>>>>>>>> DEPENDS ${PROJECT_SOURCE_DIR}/Main.java
> >>>>>>>>>>> )
> >>>>>>>>>>>
> >>>>>>>>>>> And I have a custom command that moves the generated *test1.c *
> >>>>>>>>>>> to a new directory inside the build directory.
> >>>>>>>>>>>
> >>>>>>>>>>> ADD_CUSTOM_COMMAND(
> >>>>>>>>>>> TARGET TestGen
> >>>>>>>>>>> POST_BUILD
> >>>>>>>>>>> COMMAND mv
> >>>>>>>>>>> ARGS ${PROJECT_BINARY_DIR}/test1.c ${PROJECT_BINARY_DIR}/tests/
> >>>>>>>>>>> )
> >>>>>>>>>>>
> >>>>>>>>>>> Each time I run make, the custom target is run (since custom
> >>>> targets
> >>>>>>>> are
> >>>>>>>>>>> always
> >>>>>>>>>>> out-of-date). But I want to avoid moving the new test1.c
> >> generated
> >>>>>> each
> >>>>>>>>>>> time if build/test1.c is the same as build/tests/test1.c since
> >>>> there
> >>>>>>>> are
> >>>>>>>>>>> other targets
> >>>>>>>>>>> like add_executable and add_library later in the CMakelists
> file
> >>>> that
> >>>>>>>> are
> >>>>>>>>>>> re-built
> >>>>>>>>>>> each time since they depend on test1.c
> >>>>>>>>>>
> >>>>>>>>>> Try "cmake -E copy_if_different ..."
> >>>>>>>>>>
> >>>>>>>>>> HS
> --
>
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.cmake.org/mailman/listinfo/cmake
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.cmake.org/pipermail/cmake/attachments/20120304/2fb7c5f8/attachment-0001.htm>
More information about the CMake
mailing list