[CMake] avoid rebuilding targets depending on generated source files
Ajay Panyala
ajay at csc.lsu.edu
Mon Mar 5 13:56:11 EST 2012
> if(${GrammarSource}/test.g IS_NEWER_THAN ${PROJECT_BINARY_DIR}/test.g)
> ADD_CUSTOM_TARGET(...)
> ADD_CUSTOM_COMMAND(...)
>
> i.e I am totally avoiding the process of rebuilding the test.g file and
> copying the generated files to build/parser
> *IF* test.g has not been modified.
...and if test.g is once modified, you won't have a custom target which
regenerates the source files and copies them to the parser directory -
I'm pretty sure this is not what you intend. Recall the difference
between configuration time and build time in CMakified projects.
You are right. Once test.g is modified, just running make wont rebuild it.
I have to run cmake again and then make since the IF is evaluated only
when cmake is run.
Thanks
Ajay
On Sun, Mar 4, 2012 at 7:35 PM, Michael Hertling <mhertling at online.de>wrote:
> On 03/05/2012 01:59 AM, Ajay Panyala wrote:
> > I use cmake version 2.8.3.
> >
> > If I use CMakeLists from your previous reply, it avoids overwriting files
> > when X=0.
> > I have attached the output of your script.
>
> Actually, this is exactly what I was talking about: The second "make
> X=0" invocation rewrites the source files - i.e., they receive a new
> timestamp - but their contents do not change. Thus, the destination
> files in the parser directory are *not* touched by the second "make
> X=0", and this can be seen clearly by reference to the timestamps.
> It's the subsequent "make X=1" which provides for a new content of
> the source files and makes "cmake -E copy_if_different" copy the
> sources to the destinations. That's why I said everything works
> as expected, but you wrote:
>
> >>> It does not work. The files are still overwritten.
>
> So, my question again: What exactly does not work?
>
> I.e., does "cmake -E copy_if_different"
>
> - copy a file although the destination exists
> and has the same content as the source, or
>
> - not copy a file although the destination does not
> exist or does not have the same content as the source?
>
> > It works for my CMakeLists as well now. What I did now is
> >
> > if(${GrammarSource}/test.g IS_NEWER_THAN ${PROJECT_BINARY_DIR}/test.g)
> > ADD_CUSTOM_TARGET(...)
> > ADD_CUSTOM_COMMAND(...)
> >
> > i.e I am totally avoiding the process of rebuilding the test.g file and
> > copying the generated files to build/parser
> > *IF* test.g has not been modified.
>
> ...and if test.g is once modified, you won't have a custom target which
> regenerates the source files and copies them to the parser directory -
> I'm pretty sure this is not what you intend. Recall the difference
> between configuration time and build time in CMakified projects.
>
> > PS: Could you set up your e-mail client so that it marks quoted
> > text in some way? Currently, one can't distinguish between
> > the lines you are writing and the lines written by others.
> >
> > Sorry. I did not realize that. I use gmail and when I checked my sent
> > emails,
> > quoted text appears to be clearly marked. Is the quoted text in this
> email
> > marked properly.
>
> Yes, it is.
>
> Regards,
>
> Michael
>
> > On Sun, Mar 4, 2012 at 4:05 PM, Michael Hertling <mhertling at online.de
> >wrote:
> >
> >> On 03/04/2012 08:02 PM, Ajay Panyala wrote:
> >>> The following project is a boiled-down version of yours but doesn't
> >>> need any programs except for CMake - that's what I actually meant
> >>> with "minimal but complete" as I don't have the org.antlr.Tool
> >>> Java program:
> >>>
> >>> I am sorry. I was mainly trying to cleanup the big CMakeLists file
> >>> I had and removed about 10 different targets - all of which were
> >>> rebuilt because the 5 files (test*.*) were overwritten.
> >>>
> >>> If you want to try running the test project on your system
> >>>
> >>> test.g is at http://pastie.org/private/agzor3ibzoa5pom6q31qq
> >>>
> >>> org.antlr.Tool is at www.antlr.org/download/antlr-3.4-complete.jar
> >>>
> >>> After configuration, you can run the target by "make X=0" and check the
> >>> timestamps by "ls -l --full-time test.tokens parser src". Issuing "make
> >>> X=0" again reveals that the copied files aren't rewritten as it happens
> >>> after "make X=1". Thus, AFAICS, everything works as expected. Could you
> >>> check if the above-noted example also works on your system?
> >>>
> >>> It does not work. The files are still overwritten.
> >>
> >> Could you run the following shell script in a build tree configured
> >> with the CMakeLists.txt from my previous reply and post the output?
> >>
> >> # check.sh:
> >> make X=0 2>&1 > /dev/null
> >> echo "After make X=0 (1):"
> >> echo "-------------------"
> >> echo ""
> >> ls --full-time test.tokens src parser
> >> echo -ne "\n\n"
> >> sleep 3
> >> make X=0 2>&1 > /dev/null
> >> echo "After make X=0 (2):"
> >> echo "-------------------"
> >> echo ""
> >> ls --full-time test.tokens src parser
> >> echo -ne "\n\n"
> >> sleep 3
> >> make X=1 2>&1 > /dev/null
> >> echo "After make X=1:"
> >> echo "---------------"
> >> echo ""
> >> ls --full-time test.tokens src parser
> >> # End of check.sh
> >>
> >> BTW, which CMake version do you use?
> >>
> >> Regards,
> >>
> >> Michael
> >>
> >> PS: Could you set up your e-mail client so that it marks quoted
> >> text in some way? Currently, one can't distinguish between
> >> the lines you are writing and the lines written by others.
> >>
> >>> What exactly does not work with your example? You wrote:
> >>>
> >>>>>> I have 4 cmake -E copy_if_different commands, one for each file.
> >>>>>> Only the last file is not copied (if similar). [...]
> >>>
> >>> Does this mean that the last file out of four - in fact, your example
> >>> handles five files - is not copied *although* the source file and the
> >>> destination file are different, i.e. similar but not equal?
> >>>
> >>> Yes the file test.tokens is not copied overwritten since they are
> >>> exactly (diff) similar files. This is the case with the other 4 files
> as
> >>> well,
> >>> but they are still copied over and rewritten.
> >>>
> >>>
> >>> You wrote further:
> >>>
> >>>>>> [...] The others are copied
> >>>>>> even if they are the same.
> >>>>>>
> >>>>>> I verfied that they are the same with a diff.
> >>>
> >>> Does this mean that source files are copied *although* they are equal
> >>> to their respective destination file? How do you determine that they
> >>> have been copied? Do you check the timestamps? With --full-time?
> >>>
> >>> Yes, I do check with ls -l --full-time. Except test.tokens, all the
> other
> >>> files
> >>> are copied over (rewritten) even though they are exactly the same (diff
> >>> same I mean).
> >>>
> >>> This is what is confusing me about the behavior of copy_if_different.
> >>> that is why it works only with test.tokens and not others.
> >>>
> >>> PS: Does org.antlr.Tool write to the source tree? If so: Don't do that.
> >>>
> >>> Yes, it generates the files in the source dir itself (where test.g is
> >>> present)
> >>> I now modified CMakeLists to copy test.g to the project build folder
> and
> >>> run it there. The new CMakeLists is at
> >>>
> >>> http://pastie.org/private/p1yi0l8so9cqimqlywfmhw
> >>>
> >>>
> >>> Thank You
> >>> Ajay
> >>>
> >>> On Sun, Mar 4, 2012 at 12:52 AM, Michael Hertling <mhertling at online.de
> >>> wrote:
> >>>
> >>>> On 03/04/2012 01:06 AM, Ajay Panyala wrote:
> >>>>> Please provide a minimal but complete example for this issue.
> >>>>>
> >>>>> Please find it in the following link
> >>>>> http://pastie.org/private/pd13u33s9xpfihf2dbzc1q
> >>>>
> >>>>
> >>>
> >>>
> >>>> The following project is a boiled-down version of yours but doesn't
> >>>> need any programs except for CMake - that's what I actually meant
> >>>> with "minimal but complete" as I don't have the org.antlr.Tool
> >>>> Java program:
> >>>>
> >>>> CMAKE_MINIMUM_REQUIRED(VERSION 2.8 FATAL_ERROR)
> >>>> PROJECT(P NONE)
> >>>> SET(CMAKE_VERBOSE_MAKEFILE ON)
> >>>>
> >>>> SET(GrammarSource ${PROJECT_BINARY_DIR}/src)
> >>>> FILE(MAKE_DIRECTORY ${PROJECT_BINARY_DIR}/src)
> >>>>
> >>>> ADD_CUSTOM_TARGET(GrammarBuild ALL
> >>>> COMMAND ${CMAKE_COMMAND} -E echo "\${X}"
> >>>> > ${GrammarSource}/testLexer.h
> >>>> COMMAND ${CMAKE_COMMAND} -E echo "\${X}"
> >>>> > ${GrammarSource}/testLexer.c
> >>>> COMMAND ${CMAKE_COMMAND} -E echo "\${X}"
> >>>> > ${GrammarSource}/testParser.h
> >>>> COMMAND ${CMAKE_COMMAND} -E echo "\${X}"
> >>>> > ${GrammarSource}/testParser.c
> >>>> COMMAND ${CMAKE_COMMAND} -E echo "\${X}"
> >>>> > ${PROJECT_BINARY_DIR}/test.tokens
> >>>> )
> >>>>
> >>>> ADD_CUSTOM_COMMAND(TARGET GrammarBuild POST_BUILD
> >>>> COMMAND ${CMAKE_COMMAND} -E copy_if_different
> >>>> ${GrammarSource}/testLexer.h
> >>>> ${PROJECT_BINARY_DIR}/parser/testLexer.h
> >>>> COMMAND ${CMAKE_COMMAND} -E copy_if_different
> >>>> ${GrammarSource}/testLexer.c
> >>>> ${PROJECT_BINARY_DIR}/parser/testLexer.c
> >>>> COMMAND ${CMAKE_COMMAND} -E copy_if_different
> >>>> ${GrammarSource}/testParser.h
> >>>> ${PROJECT_BINARY_DIR}/parser/testParser.h
> >>>> COMMAND ${CMAKE_COMMAND} -E copy_if_different
> >>>> ${GrammarSource}/testParser.c
> >>>> ${PROJECT_BINARY_DIR}/parser/testParser.c
> >>>> COMMAND ${CMAKE_COMMAND} -E copy_if_different
> >>>> ${PROJECT_BINARY_DIR}/test.tokens
> >>>> ${PROJECT_BINARY_DIR}/parser/test.tokens
> >>>> )
> >>>>
> >>>> After configuration, you can run the target by "make X=0" and check
> the
> >>>> timestamps by "ls -l --full-time test.tokens parser src". Issuing
> "make
> >>>> X=0" again reveals that the copied files aren't rewritten as it
> happens
> >>>> after "make X=1". Thus, AFAICS, everything works as expected. Could
> you
> >>>> check if the above-noted example also works on your system?
> >>>>
> >>>> What exactly does not work with your example? You wrote:
> >>>>
> >>>>>>> I have 4 cmake -E copy_if_different commands, one for each file.
> >>>>>>> Only the last file is not copied (if similar). [...]
> >>>>
> >>>> Does this mean that the last file out of four - in fact, your example
> >>>> handles five files - is not copied *although* the source file and the
> >>>> destination file are different, i.e. similar but not equal?
> >>>>
> >>>> You wrote further:
> >>>>
> >>>>>>> [...] The others are copied
> >>>>>>> even if they are the same.
> >>>>>>>
> >>>>>>> I verfied that they are the same with a diff.
> >>>>
> >>>> Does this mean that source files are copied *although* they are equal
> >>>> to their respective destination file? How do you determine that they
> >>>> have been copied? Do you check the timestamps? With --full-time?
> >>>>
> >>>> Regards,
> >>>>
> >>>> Michael
> >>>>
> >>>> PS: Does org.antlr.Tool write to the source tree? If so: Don't do
> that.
> >>>>
> >>>>> On Sat, Mar 3, 2012 at 3:54 PM, Michael Hertling <
> mhertling at online.de
> >>>>> wrote:
> >>>>>
> >>>>>> On 03/04/2012 12:14 AM, Ajay Panyala wrote:
> >>>>>>> No, it wouldn't; check it out:
> >>>>>>>
> >>>>>>> % touch a
> >>>>>>> % rm -f b
> >>>>>>> % ls b
> >>>>>>> ls: cannot access b: No such file or directory
> >>>>>>> % cmake -E copy_if_different a b
> >>>>>>> % ls b
> >>>>>>> b
> >>>>>>> % cksum a b
> >>>>>>> 4294967295 0 a
> >>>>>>> 4294967295 0 b
> >>>>>>>
> >>>>>>> It works with one file, but I have 4 files that are generated.
> >>>>>>> I have 4 cmake -E copy_if_different commands, one for each file.
> >>>>>>> Only the last file is not copied (if similar). The others are
> copied
> >>>>>>> even if they are the same.
> >>>>>>>
> >>>>>>> I verfied that they are the same with a diff.
> >>>>>>>
> >>>>>>> Any idea what might be happening here ?
> >>>>>>
> >>>>>> Please provide a minimal but complete example for this issue.
> >>>>>>
> >>>>>> Regards,
> >>>>>>
> >>>>>> Michael
> >>>>>>
> >>>>>>> On Sat, Mar 3, 2012 at 2:47 PM, Michael Hertling <
> >> mhertling at online.de
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> On 03/03/2012 10:36 PM, Ajay Panyala wrote:
> >>>>>>>>> Try "cmake -E copy_if_different ..."
> >>>>>>>>>
> >>>>>>>>> cmake -E copy_if_different build/test1.c build/tests/test1.c
> >>>>>>>>>
> >>>>>>>>> That would work when make is run atleast once.
> >>>>>>>>> When running make for the 1st time test1.c was never
> >>>>>>>>> copied to build/tests before. So I would be comparing a file with
> >>>>>>>>> another non-existant file and that would result in an error
> halting
> >>>>>>>>> the make process.
> >>>>>>>>
> >>>>>>>> No, it wouldn't; check it out:
> >>>>>>>>
> >>>>>>>> % touch a
> >>>>>>>> % rm -f b
> >>>>>>>> % ls b
> >>>>>>>> ls: cannot access b: No such file or directory
> >>>>>>>> % cmake -E copy_if_different a b
> >>>>>>>> % ls b
> >>>>>>>> b
> >>>>>>>> % cksum a b
> >>>>>>>> 4294967295 0 a
> >>>>>>>> 4294967295 0 b
> >>>>>>>>
> >>>>>>>> Regards,
> >>>>>>>>
> >>>>>>>> Michael
> >>>>>>>>
> >>>>>>>>> On Sat, Mar 3, 2012 at 1:20 PM, Hendrik Sattler <
> >>>>>> post at hendrik-sattler.de
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Am Samstag, 3. März 2012, 21:41:49 schrieb Ajay Panyala:
> >>>>>>>>>>> I have a custom target which runs a command to generate
> >>>>>>>>>>> a C source file say test1.c
> >>>>>>>>>>>
> >>>>>>>>>>> ADD_CUSTOM_TARGET(TestGen ALL
> >>>>>>>>>>> COMMAND genExec ${PROJECT_SOURCE_DIR}/Main.java
> >>>>>>>>>>> DEPENDS ${PROJECT_SOURCE_DIR}/Main.java
> >>>>>>>>>>> )
> >>>>>>>>>>>
> >>>>>>>>>>> And I have a custom command that moves the generated *test1.c *
> >>>>>>>>>>> to a new directory inside the build directory.
> >>>>>>>>>>>
> >>>>>>>>>>> ADD_CUSTOM_COMMAND(
> >>>>>>>>>>> TARGET TestGen
> >>>>>>>>>>> POST_BUILD
> >>>>>>>>>>> COMMAND mv
> >>>>>>>>>>> ARGS ${PROJECT_BINARY_DIR}/test1.c ${PROJECT_BINARY_DIR}/tests/
> >>>>>>>>>>> )
> >>>>>>>>>>>
> >>>>>>>>>>> Each time I run make, the custom target is run (since custom
> >>>> targets
> >>>>>>>> are
> >>>>>>>>>>> always
> >>>>>>>>>>> out-of-date). But I want to avoid moving the new test1.c
> >> generated
> >>>>>> each
> >>>>>>>>>>> time if build/test1.c is the same as build/tests/test1.c since
> >>>> there
> >>>>>>>> are
> >>>>>>>>>>> other targets
> >>>>>>>>>>> like add_executable and add_library later in the CMakelists
> file
> >>>> that
> >>>>>>>> are
> >>>>>>>>>>> re-built
> >>>>>>>>>>> each time since they depend on test1.c
> >>>>>>>>>>
> >>>>>>>>>> Try "cmake -E copy_if_different ..."
> >>>>>>>>>>
> >>>>>>>>>> HS
> --
>
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.cmake.org/mailman/listinfo/cmake
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.cmake.org/pipermail/cmake/attachments/20120305/e20be4e0/attachment-0001.htm>
More information about the CMake
mailing list