[CMake] Problem with generated source and header files

Fri Jul 3 09:50:09 EDT 2009

I'm very happy to announce, that I have found a solution.

Problem was:
Either CMake stopped, because a dependend source file was missing (not generated yet).
Or 'make' ALWAYS tried to create the generated source files multiple times, because I had to use ADD_CUSTOM_TARGET, because the generation process is done in a separate directory.

Solution:
I realized that I can split the process of generating the source files into two phases:
Phase 1: Initial generation (while 'cmake' runs)
Phase 2: Generation based on dependencies (while 'make' runs)
The good thing is now that I don't have to check for dependencies when initially creating all the sources.
CMake does not stop any more. All source files are there.

In each file, which uses generated sources, I add an include file "generated.cmake" to create
- the source and header files
- A file "generated_srcs.cmake" to be included in CMakeLists.txt. This assigns a all source file names to a variable GENERATED_SRCS.

The good thing of having "generated.cmake" is, that I don't have to use ADD_CUSTOM_TARGET() to make the outputs known my current CMakeLists.txt file. By being able to use ADD_CUSTOM_COMMAND(OUTPUT...), the sources are not generated ALWAYS (like before, because ADD_CUSTOM_TARGET() is always considered out of date)

The good thing of having "generated_srcs.cmake" is, that I now have a list of source file names to be used to build the library and make dependencies. I don't need file globbing any more and CMake does not complain, that "${GENERATED_SRCS} is empty and there is probably something wrong with my CMakeLists.txt file"

Additionally I use a file, which is generated, as a flag.
If the file ${CMAKE_BINARY_DIR}/imt/build_mib is not found, the generator just creates all files (initial).

Part of CMakeLists.txt:
----
# Create generated source and header files (e.g. for MIB, CFG and SOAP)
INCLUDE( ${CMAKE_SOURCE_DIR}/imt/generated.cmake )

# Include list of generated files to build the library
INCLUDE( ${CMAKE_CURRENT_BINARY_DIR}/generated/generated_srcs.cmake )

# Define a target shared object library
ADD_LIBRARY( mibfactory SHARED ${GENERATED_SRCS} )

# Add dependencies between library and generated files
ADD_DEPENDENCIES( mibfactory build_mib )
----

Part of imt/generated.cmake:
----
add_custom_command(
  OUTPUT  ${CMAKE_BINARY_DIR}/imt/build_mib
  COMMAND ruby imt.rb ${IMDL_XML} ${MIB_NS} ${ROOT}
  DEPENDS ${RUBY_FILES}
  DEPENDS ${MIB_RUBY_FILES}
  WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}/imt
  VERBATIM
)

# Initially create generated files, if they don't yet exist
IF( NOT EXISTS ${CMAKE_BINARY_DIR}/build_mib )
  MESSAGE( "*** Create generated MIB files ***" )
  EXECUTE_PROCESS(
    COMMAND ruby imt.rb ${IMDL_XML} ${MIB_NS} ${ROOT}
    WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}/imt
  )
ENDIF()
----

Example for generated_srcs.cmake:
----
GENERATED_SRCS(
  file_hjkfdk.cpp
  file_kjwerui.cpp
  file_wqeierp.cpp
  ...
)
----

Thanks a lot for your help.

Best regards,
Joerg

> -----Ursprüngliche Nachricht-----
> Von: Marcel Loose [mailto:loose at astron.nl] 
> Gesendet: Donnerstag, 2. Juli 2009 12:13
> An: cmake at cmake.org
> Cc: Jörg Förstner
> Betreff: Re: [CMake] Problem with generated source and header files
> 
> Hi Joerg,
> 
> As far as I know there's no hook in CMake to do this. Did you 
> try Eric's 
> suggestion already?
> 
> Best regards,
> Marcel Loose.
> 
> 
> On Thursday 02 July 2009 10:51:23 Jörg Förstner wrote:
> > Hi,
> >
> > > Anyway, IMHO, the effort of keeping a list of source files
> > > up-to-date is much
> > > less than writing the content of these sources files.
> >
> > Keeping/generating the list of source files is easy, that's not the
> > problem.
> >
> > The problem is, _when_ the list can be generated.
> > And _when_ cmake includes/processes this list.
> >
> > If the list is generated in the cmake process, there seems 
> to be no way of
> > re-importing it within the cmake run. An automated 2-pass 
> cmake/make run
> > also seems not to be possible.
> > (manually running cmake/make two times is not desired)
> >
> > Is there a possibility to do things in between cmake evaluating the
> > CMakeLists.txt files and cmake executing the commands in 
> the CMakeLists.txt
> > files (some kind of a "hook")?
> >
> > Regards,
> > Joerg
> >
> > > -----Ursprüngliche Nachricht-----
> > > Von: cmake-bounces at cmake.org [mailto:cmake-bounces at cmake.org]
> > > Im Auftrag von Marcel Loose
> > > Gesendet: Donnerstag, 2. Juli 2009 09:57
> > > An: cmake at cmake.org
> > > Betreff: Re: [CMake] Problem with generated source and 
> header files
> > >
> > > Hi James,
> > >
> > > On Wednesday 01 July 2009 15:58:12 James C. Sutherland wrote:
> > > > > So, in general, when using globbing, YOU are responsible
> > >
> > > for rerunning
> > >
> > > > > CMake whenever you've added a source file. Otherwise 
> you run the
> > > > > risk of
> > > > > the new file not being compiled. Furthermore, you might
> > >
> > > accidentally
> > >
> > > > > compile sources that were just lying around in your
> > >
> > > directory as test
> > >
> > > > > code. Deletion of sources can also cause interesting
> > >
> > > effects if you
> > >
> > > > > create a library, because the object will remain in that
> > >
> > > library until
> > >
> > > > > you (manually) remove and recreate the library.
> > > > >
> > > > > I hope my examples convinced you enough that globbing is (in
> > > > > general) a
> > > > > bad idea.
> > > >
> > > > So is it common practice among users of CMake to 
> manually create and
> > > > maintain a list of all files that are to be compiled, 
> even if such a
> > > > list is very large and may involve several directories and
> > > > subdirectories?
> > >
> > > I can't talk for others, but I think it's more the rule than
> > > the exception.
> > > But no-one forbids you to use globbing. It's just that you
> > > should be aware of
> > > the consequences; things can go wrong in very subtle ways.
> > >
> > > Anyway, IMHO, the effort of keeping a list of source files
> > > up-to-date is much
> > > less than writing the content of these sources files.
> > >
> > > Just my 2 cents.
> > >
> > > Regards,
> > > Marcel Loose
> > > _______________________________________________
> > > Powered by www.kitware.com
> > >
> > > Visit other Kitware open-source projects at
> > > http://www.kitware.com/opensource/opensource.html
> > >
> > > Please keep messages on-topic and check the CMake FAQ at:
> > > http://www.cmake.org/Wiki/CMake_FAQ
> > >
> > > Follow this link to subscribe/unsubscribe:
> > > http://www.cmake.org/mailman/listinfo/cmake
> >
> > _______________________________________________
> > Powered by www.kitware.com
> >
> > Visit other Kitware open-source projects at
> > http://www.kitware.com/opensource/opensource.html
> >
> > Please keep messages on-topic and check the CMake FAQ at:
> > http://www.cmake.org/Wiki/CMake_FAQ
> >
> > Follow this link to subscribe/unsubscribe:
> > http://www.cmake.org/mailman/listinfo/cmake
> 
> 