CMake Fortran Issues: Difference between revisions

From KitwarePublic
Jump to navigationJump to search
(Replace content with link to new CMake community wiki)
 
(2 intermediate revisions by one other user not shown)
Line 1: Line 1:
__TOC__
{{CMake/Template/Moved}}


= Introduction to this wikipage =
This page has moved [https://gitlab.kitware.com/cmake/community/wikis/doc/cmake/languages/fortran/Fortran-Issues here].
CMake has a number of Fortran issues that have been discussed many different times on list and duplicated a fair number of times in the bug tracker as well.
 
Maik Beckmann is trying to make sense of all the confusion by collecting information on all Fortran issues at http://www.cmake.org/Bug/view.php?id=5809
 
Please join the work there by
 
* Contributing patches.
* Testing the patches that already exist there.
* Reporting things that don't work.
* Sending simplified examples of things which don't work.
* Sharing your expert knowledge of CMake.
 
= Introduction to the problem=
 
Hello CMake developers,
 
I pushed myself during the last weekends to get more familiar with CMakes
codebase.  Not for fun only ;), but make me smart enough to sketch an
approach for handling fortrans module dependencies.
 
I will try to write down what I'm thinking about
 
To give you (and me) some orientation here are the steps which are done by
cmake so far and what I'm going to plan.
 
CMakes atom of dependency is a sourcefile.  It knows the language of the sourcefile and handles it by the related class
cmDepends**
When CMake processes foo/.c/.cxx/.java/.f/.f90 it doesn't know anything about the CONTENT! of any other sourcefile.  This is ok for include dependencies,  since the included file is almost somewhere at the disk. 
 
Anyway, this is not sufficient for fortran. If a fortran source foo.f90 contains something like
<code>
  module bar
    ...
  end module
</code>
the compiler will generate a file called foo.mod. A .mod file can be considered as a __dynamically generated header file__.  Every fortran source can potentially create a .mod file!  So if cmake processes a fortran sourcefile which needs bar.mod it know that foo.f90 provides this module.
 
= Abstract solution =
 
The Plan is to scan all fortran source in the sourcetree before the per file dependency generation is done.  The extracted information are serialized.  When cmake processes a fortran source it know if a required module is provided by
- a source of the current target
- a source of another target in the source tree
- if none of both it must be part of an prebuild library
 
: '''Brad''': At a high level this approach looks correct.  The goal is to get sources using modules to rebuild when the sources providing them have changed.  We may not be able to magically add inter-target dependencies automatically.  However as we've discussed before that is probably okay because the library providing a module must be linked into targets using it anyway.
 
: '''Brad''': The other thing for which we need to watch out is generated fortran sources.  What if there is a custom command generating a fortran source that provides or uses a module?  This may be a bit too much to expect.
 
= Concrete solution =
 
 
Current CMake does:
After cmake ran once of the source tree (i.e cmake -G"Unix
Makefiles" /path/to/sourcetree)
<pre>
1. check build system
...
k    : $(MAKE) -f  ../foo.dir/build.make ../foo.dir/depend
k+1: $(MAKE) -f ../foo.dir/build.make  ../foo.dir/requires
k+2: $(MAKE) -f ../foo.dir/build.make  ../foo.dir/build
...
l    : $(MAKE) -f  ../bar.dir/build.make ../bar.dir/depend
l+1: $(MAKE) -f ../bar.dir/build.make  ../bar.dir/requires
l+1: $(MAKE) -f ../bar.dir/build.make  ../bar.dir/build
</pre>
 
: '''Brad''': FYI, I've wanted to change this design a bit unrelated to fortran support.  The check-build-system step currently does work globally when most of that work could be done on a per-target basis.  Fixing this problem will allow faster make times for building individual targets and their dependencies.  I'll have to think about where the fortran scanning step belongs in this process.
 
It would be cool if it will do:
<pre>
1  : check build system
...
# maybe only for targets which have fortran sources
j    : $(MAKE) -f    ../foo.dir/build.make ../foo.dir/fortran_module_scan
j+1: $(MAKE) -f    ../bar.dir/build.make ../bar.dir/fortran_module_scan
...
k    : $(MAKE) -f  ../foo.dir/build.make ../foo.dir/depend
k+1: $(MAKE) -f ../foo.dir/build.make  ../foo.dir/build
...
l    : $(MAKE) -f  ../bar.dir/build.make ../bar.dir/depend
l+1: $(MAKE) -f ../bar.dir/build.make  ../bar.dir/build
</pre>
.dir/fortran_module_scan
will trigger a new cmake command
$(CMAKE_COMMAND) -E fortran_module_scan ...
while .dir/depend triggers still tiggers
$(CMAKE_COMMAND) -E cmake_depends ...
 
: '''Brad''': We'll have to make sure that all the fortran scanning is done for all targets before the per-target depend.make generation is done.
 
== -E fortran_module_scan  ==
This is a new cmake command.
 
The information which is extracted from a source "foo.f90" are
- required modules
- provided modules
- includes
These three vectors and the name "foo.f90" are stored in a data structure
called "FortranSourceDependInfo".
<pre>
struct cmFortranSourceDependInfo
{
  std::string Name;
  std::vector<std::string> Requires;
  std::vector<std::string> Provides;
  std::vector<std::string> Includes;
  // std::set needs this operator
  bool operator<(cmFortranSourceDependInfo const& rhs) const
  {  this->Name < rhs.Name; }
};
</pre>
This happens for every source of a Target i.e. "mylib".  Another data
structure "FortranTargetDependInfo"  holds the name "mylib" and a set
of "FortranSourceDependInfo" instances. 
<pre>
struct cmFortranTargetDependInfo
{
  std::string Name;
  std::set<cmFortranSourceDependInfo> Sources;
  void GetModuleInfoOfSource(char const* src,  cmFortranSourceDependInfo&)
    const;
  // does this target source provide module mod?
  bool Provides(char const* mod) const;
};
</pre>
This "cmFortranTargetDependInfo" instance is finally stored into
<pre>
class cmFortranDependInfos
{
public:
  typedef std::map<std::string, cmFortranTargetDependInfo > targets_type;
 
  cmFortranDependInfos(char const* homeDir);
  virtual ~cmFortranDependInfos();
 
  void InsertSourceDependInfo(char const* targetName,
    cmFortranSourceDependInfo const&);
 
  void GetModuleInfoOfTarget(char const* targetName,
    cmFortranTargetDependInfo&) const;
 
  void Serialize();
  void Load(char const* homeDir);
 
private:
  static char const* ArchiveName;
  targets_type Targets;
  std::string Archive;
};
</pre>
This is now serialized to a file at the build directory (at the moment I'm using
boost.serialization. Are there serialization capabilities in CMake?)
 
: '''Brad''': Instead of serialization we usually encode the information in a .cmake file that is later loaded with ReadListFile.
 
== -E cmake_depends ==
One of the very first things which happens when -E cmake_depends is executed,
is the construction of an "class cmake" instance, which itself constructs a
cmGlobalGenerator.  cmake#CreateGlobalGenerator(..) looks for an the
serialized cmFortranDependInfos instance and loads it.  Now the per
sourcefile dependency generation scheme of cmake knows enough to do the right
thing for fortran modules.
 
: '''Brad''': Some kind of scheme for this step to know where to load the fortran information will be needed.  Global information must be available for each target.
 
= OUTDATED: Concepts expressed using Makefiles =
 
This section is intended to discuss the Makefile rules which CMake has to generate.  All examples are fully working. You can download them as tarball examples_using_Makefiles.tar.gz at http://www.cmake.org/Bug/view.php?id=5809. To build an example, change into the corresponding ''build'' directory and run the
: $ make
command.  After this initial build, check dependencies by touching source files of your choice and running the
: $ make
command again.
 
'''Note:''' For examples which show how an external library providing modules is handled, the external library which resides at directory ''extLib'' for each of these examples has to be built and installed by changing into the corresponding ''extLib'' directory and running the
: $ make install && make clean
command.
 
== A simple program ==
 
A f9x program which is build by compiling in linking two source files ''a.f90'' and ''main.f90''.
The tree structure is:
* example_simpleProgram
** build
***  Makefile
*** prog.dir
**** build.make
** main.f90
** a.f90
 
a.f90:
<pre>
SUBROUTINE printHello
    WRITE(*,*) "Hello f9x world"
END SUBROUTINE
</pre>
 
main.f90:
<pre>
PROGRAM hello
    CALL printHello
END PROGRAM
</pre>
 
Makefile:
<pre>
all: prog.dir/all
 
prog.dir/all:
$(MAKE) -f prog.dir/build.make prog.dir/all
 
clean:
$(MAKE) -f prog.dir/build.make prog.dir/clean
</pre>
 
build.make:
<pre>
prog.dir/all: prog.dir/prog
 
prog.dir/prog:  prog.dir/a.o prog.dir/main.o
gfortran -o prog.dir/prog  prog.dir/a.o prog.dir/main.o
prog.dir/a.o: ../a.f90
gfortran -o prog.dir/a.o  -c ../a.f90
prog.dir/main.o: ../main.f90
gfortran -o prog.dir/main.o  -c ../main.f90
prog.dir/clean:
rm prog.dir/a.o prog.dir/main.o prog.dir/prog
</pre>
 
The rules generated by the '''current CMake covers all dependencies''' which can occur as long as '''no modules''' are used.
 
You can download this example as tarball example_simpleProgram.tar.gz at http://www.cmake.org/Bug/view.php?id=5809
 
== A simple program with module ==
 
The same as before, but now ''a.f90'' provides a module which ''main.f90'' uses.
The tree structure is:
* example_simpleProgram_withModule
** build
***  Makefile
*** prog.dir
**** build.make
** main.f90
** a.f90
 
a.f90:
<pre>
MODULE localMod
!
CONTAINS
    SUBROUTINE printHello
        WRITE(*,*) "Hello f9x world"
    END SUBROUTINE
END MODULE
</pre>
 
main.f90:
<pre>
PROGRAM hello
    USE localMod
    CALL printHello
END PROGRAM
</pre>
 
=== Rules like those generated by current CMake ===
 
Makefile:
<pre>
all: prog.dir/all
 
prog.dir/all:
$(MAKE) -f prog.dir/build.make prog.dir/requires
$(MAKE) -f prog.dir/build.make prog.dir/all
 
clean:
$(MAKE) -f prog.dir/build.make prog.dir/clean
</pre>
 
build.make:
<pre>
prog.dir/all: prog.dir/prog
 
 
prog.dir/prog:  prog.dir/a.o prog.dir/main.o
gfortran -o prog.dir/prog  prog.dir/a.o prog.dir/main.o
 
prog.dir/a.o: ../a.f90
gfortran -o prog.dir/a.o  -c ../a.f90 -M prog.dir
prog.dir/localmod.mod: prog.dir/a.o
prog.dir/main.o: ../main.f90
gfortran -o prog.dir/main.o  -c ../main.f90 -I prog.dir
prog.dir/clean:
rm prog.dir/localmod.mod prog.dir/a.o prog.dir/main.o prog.dir/prog
localmod.mod.proxy: prog.dir/a.o
 
prog.dir/main.o.requires: localmod.mod.proxy
prog.dir/requires: prog.dir/main.o.requires
</pre>
 
After you build prog using this set of Makefiles
do (you're at the build directory)
  $ touch ../a.f90
and enter
  $ make
You'll see that a.f90 is recompiled and prog.dir/prog  is linked again. But main.f90 has to recompiled too, since a module dependency is a compile time dependency like an include.
 
=== Rules like those that should be generated by CMake ===
 
Makefile:
<pre>
all: prog.dir/all
 
prog.dir/all:
$(MAKE) -f prog.dir/build.make prog.dir/all
 
clean:
$(MAKE) -f prog.dir/build.make prog.dir/clean
</pre>
 
build.make:
<pre>
prog.dir/all: prog.dir/prog
 
 
prog.dir/prog:  prog.dir/a.o prog.dir/main.o
gfortran -o prog.dir/prog  prog.dir/a.o prog.dir/main.o
 
prog.dir/a.o: ../a.f90
gfortran -o prog.dir/a.o  -c ../a.f90 -M prog.dir
prog.dir/localmod.mod: prog.dir/a.o
prog.dir/main.o: ../main.f90 prog.dir/localmod.mod
gfortran -o prog.dir/main.o  -c ../main.f90 -I prog.dir
prog.dir/clean:
rm prog.dir/localmod.mod prog.dir/a.o prog.dir/main.o prog.dir/prog
</pre>
 
After you build prog using this set of Makefiles
do (you're at the build directory)
  $ touch ../a.f90
and enter
  $ make
You'll see that a.f90 is recompiled, like the current CMake does,
but main.f90  is recompiled too, as it should be.
 
== Executable depending on external lib ==
 
This example build a executable target which
# provides a module
# uses the provided module
# uses a module of a external library
 
structure:
* example_dependingOn_externalLib
** extLib
*** include
**** externalmod.mod
*** lib
**** libmyextlib.a
** myproject
*** build
**** Makefile
**** prog.dir
***** build.make
*** a.f90
*** main.f90
 
Contents of myproject...
 
a.f90:
<pre>
MODULE localMod
!
CONTAINS
    SUBROUTINE printLocalModGreeting
        WRITE(*,*) "Greetings from Module localMod"
    END SUBROUTINE
END MODULE
</pre>
 
main.f90:
<pre>
PROGRAM hello
    USE localMod
    USE externalMod
    CALL printLocalModGreeting
    CALL printExtModGreeting
END PROGRAM
</pre>
 
=== Rules like those generated by current CMake ===
Makefile:
<pre>
all: prog.dir/all
 
prog.dir/all:
$(MAKE) -f prog.dir/build.make prog.dir/requires
$(MAKE) -f prog.dir/build.make prog.dir/all
 
clean:
$(MAKE) -f prog.dir/build.make prog.dir/clean
</pre>
 
build.make:
<pre>
prog.dir/all: prog.dir/prog
 
 
prog.dir/prog: ../../extLib/lib/libmyextlib.a
prog.dir/prog: prog.dir/a.o prog.dir/main.o
gfortran -o prog.dir/prog  prog.dir/a.o prog.dir/main.o ../../extLib/lib/libmyextlib.a
 
prog.dir/a.o: ../a.f90
gfortran -o prog.dir/a.o  -c ../a.f90 -M prog.dir
prog.dir/main.o: ../main.f90
gfortran -o prog.dir/main.o -c ../main.f90 -I prog.dir -I ../../extLib/include
prog.dir/clean:
rm prog.dir/localmod.mod prog.dir/a.o prog.dir/main.o prog.dir/prog
externalmod.mod.proxy: # dummy
 
localmod.mod.proxy: prog.dir/a.o
 
prog.dir/main.o.requires: localmod.mod.proxy externalmod.mod.proxy
prog.dir/requires: prog.dir/main.o.requires
</pre>
 
This rules got the same problem as the example above (simple Program with module) plus it doesn't recognizes
if the external modules got updated.
 
=== Rules like those that should be generated by CMake ===
 
Makefile:
<pre>
all: prog.dir/all
 
prog.dir/all:
$(MAKE) -f prog.dir/build.make prog.dir/all
 
clean:
$(MAKE) -f prog.dir/build.make prog.dir/clean
</pre>
 
build.make:
<pre>
prog.dir/all: prog.dir/prog
 
 
prog.dir/prog: ../../extLib/lib/libmyextlib.a
prog.dir/prog: prog.dir/a.o prog.dir/main.o
gfortran -o prog.dir/prog  prog.dir/a.o prog.dir/main.o ../../extLib/lib/libmyextlib.a
 
prog.dir/a.o: ../a.f90
gfortran -o prog.dir/a.o  -c ../a.f90 -M prog.dir
prog.dir/localmod.mod: prog.dir/a.o
prog.dir/main.o: ../main.f90 prog.dir/localmod.mod
gfortran -o prog.dir/main.o -c ../main.f90 -I prog.dir -I ../../extLib/include
prog.dir/clean:
rm prog.dir/localmod.mod prog.dir/a.o prog.dir/main.o prog.dir/prog
</pre>
 
These rules build everything in proper order and consider the timestamp of externalmod.mod.
 
== Executable target depending on lib target ==
 
structure:
* example_depending_libTarget
** build
*** Makefile
*** lib.dir
**** build.make
**** libmodx.mod.stamp
**** libmody.mod.stamp
*** prog.dir
**** build.make
** lib
*** a.f90
*** b.f90
** prog
*** a.f90
*** main.f90
 
contents...
 
lib/a.f90:
<pre>
MODULE libModX
    USE libModY
END MODULE
</pre>
 
lib/b.f90:
<pre>
MODULE libModY
END MODULE
</pre>
 
prog/a.f90:
<pre>
MODULE localMod
END MODULE
</pre>
 
prog/main.f90:
<pre>
PROGRAM hello
    USE localMod
    USE libModX
 
    WRITE(*,*) 'Hello, F90 world.'
END PROGRAM
</pre>
 
===  Rules like those generated by current CMake ===
 
build/Makefile:
<pre>
all: lib.dir/all prog.dir/all
 
lib.dir/all:
$(MAKE) -f lib.dir/build.make lib.dir/requires
$(MAKE) -f lib.dir/build.make lib.dir/all
 
prog.dir/all: lib.dir/all
$(MAKE) -f prog.dir/build.make  prog.dir/requires
$(MAKE) -f prog.dir/build.make  prog.dir/all
 
clean:
$(MAKE) -f prog.dir/build.make prog.dir/clean
$(MAKE) -f lib.dir/build.make lib.dir/clean
</pre>
 
build/lib.dir/build.make:
<pre>
lib.dir/all: lib.dir/mylib
 
lib.dir/mylib: lib.dir/libmylib.a
 
lib.dir/libmylib.a: lib.dir/a.o lib.dir/b.o
ar rc lib.dir/libmylib.a lib.dir/a.o lib.dir/b.o 
ranlib lib.dir/libmylib.a
 
lib.dir/a.o: ../lib/a.f90
gfortran -o lib.dir/a.o -c ../lib/a.f90 -M lib.dir
 
lib.dir/b.o: ../lib/b.f90
gfortran -o lib.dir/b.o -c ../lib/b.f90  -M lib.dir
 
libmody.mod.proxy: lib.dir/b.o
 
lib.dir/a.o.requires: libmody.mod.proxy
lib.dir/requires:  lib.dir/a.o.requires 
 
 
lib.dir/clean:
rm lib.dir/a.o lib.dir/b.o lib.dir/libmylib.a
rm lib.dir/libmodx.mod lib.dir/libmody.mod
</pre>
 
build/prog.dir/build.make:
<pre>
prog.dir/all: prog.dir/prog
 
 
prog.dir/prog: prog.dir/main.o prog.dir/a.o
gfortran -o prog.dir/prog  prog.dir/main.o prog.dir/a.o  lib.dir/libmylib.a
 
prog.dir/a.o: ../prog/a.f90
gfortran -o prog.dir/a.o -c ../prog/a.f90 -M prog.dir
prog.dir/localmod.mod: prog.dir/a.o
 
 
prog.dir/main.o: ../prog/main.f90
gfortran -o prog.dir/main.o -c ../prog/main.f90 -I lib.dir -I prog.dir
 
 
localmod.mod.proxy: prog.dir/a.o
libmodx.mod.proxy: # dummy
 
prog.dir/main.o.requires: localmod.mod.proxy libmodx.mod.proxy
prog.dir/requires: prog.dir/main.o.requires
 
 
prog.dir/clean:
rm prog.dir/a.o prog.dir/main.o prog.dir/prog prog.dir/localmod.mod
</pre>
 
Again everything is build, but isn't updated proper.
 
=== Rules like those that should be generated by CMake ===
 
build/Makefile:
<pre>
all: lib.dir/all prog.dir/all
 
lib.dir/all:
$(MAKE) -f lib.dir/build.make lib.dir/all
 
prog.dir/all: lib.dir/all
$(MAKE) -f prog.dir/build.make prog.dir/all
</pre>
 
build/lib.dir/build.make:
<pre>
lib.dir/all: lib.dir/mylib
 
lib.dir/mylib: lib.dir/libmylib.a
 
lib.dir/libmylib.a: lib.dir/a.o lib.dir/b.o
ar rc lib.dir/libmylib.a lib.dir/a.o lib.dir/b.o 
ranlib lib.dir/libmylib.a
 
lib.dir/a.o: ../lib/a.f90 lib.dir/libmody.mod
gfortran -o lib.dir/a.o -c ../lib/a.f90 -M lib.dir
touch lib.dir/libmodx.mod.stamp
lib.dir/b.o: ../lib/b.f90
gfortran -o lib.dir/b.o -c ../lib/b.f90  -M lib.dir
touch lib.dir/libmody.mod.stamp
 
lib.dir/libmodx.mod: lib.dir/a.o
lib.dir/libmody.mod: lib.dir/b.o
 
 
lib.dir/clean:
rm lib.dir/a.o lib.dir/b.o lib.dir/libmylib.a
rm lib.dir/libmodx.mod lib.dir/libmody.mod
</pre>
 
build/prog.dir/build.make:
<pre>
prog.dir/all: prog.dir/prog
 
 
prog.dir/prog: prog.dir/main.o prog.dir/a.o
gfortran -o prog.dir/prog  prog.dir/main.o prog.dir/a.o  lib.dir/libmylib.a
 
prog.dir/a.o: ../prog/a.f90
gfortran -o prog.dir/a.o -c ../prog/a.f90 -M prog.dir
prog.dir/localmod.mod: prog.dir/a.o
 
prog.dir/main.o: lib.dir/libmodx.mod.stamp
prog.dir/main.o: ../prog/main.f90 prog.dir/localmod.mod
gfortran -o prog.dir/main.o -c ../prog/main.f90 -I lib.dir -I prog.dir
 
 
 
prog.dir/clean:
rm prog.dir/a.o prog.dir/main.o prog.dir/prog prog.dir/localmod.mod
</pre>
 
== Finally: Executable target depending on lib target and external lib ==
 
structure:
* example_final
** extLib
*** include
**** externalmod.mod
*** lib
**** libmyextlib.a
** myproject
*** build
**** Makefile
**** lib.dir
***** build.make
***** libmodx.mod.stamp
***** libmody.mod.stamp
**** prog.dir
***** build.make
*** lib
**** a.f90
**** b.f90
*** prog
**** a.f90
**** main.f90
 
 
 
Contents...
 
lib/a.f90:
<pre>
MODULE libModX
    USE libModY
END MODULE
</pre>
 
lib/b.f90:
<pre>
MODULE libModY
END MODULE
</pre>
 
prog/a.f90:
<pre>
MODULE localMod
END MODULE
</pre>
 
 
prog/b.f90:
<pre>
PROGRAM hello
    USE localMod
    USE libModX
    USE externalMod
 
    WRITE(*,*) 'Hello, F90 world.'
    CALL printExtModGreeting
END PROGRAM
</pre>
 
 
=== Rules like those generated by current CMake ===
 
build/Makefile:
<pre>
all: lib.dir/all prog.dir/all
 
lib.dir/all:
$(MAKE) -f lib.dir/build.make lib.dir/requires
$(MAKE) -f lib.dir/build.make lib.dir/all
 
prog.dir/all: lib.dir/all
$(MAKE) -f prog.dir/build.make prog.dir/requires
$(MAKE) -f prog.dir/build.make prog.dir/all
 
clean:
$(MAKE) -f prog.dir/build.make prog.dir/clean
$(MAKE) -f lib.dir/build.make lib.dir/clean
</pre>
 
build/lib.dir/build.make:
<pre>
lib.dir/all: lib.dir/mylib
 
lib.dir/mylib: lib.dir/libmylib.a
 
lib.dir/libmylib.a: lib.dir/a.o lib.dir/b.o
ar rc lib.dir/libmylib.a lib.dir/a.o lib.dir/b.o 
ranlib lib.dir/libmylib.a
 
lib.dir/a.o: ../lib/a.f90
gfortran -o lib.dir/a.o -c ../lib/a.f90 -M lib.dir
lib.dir/b.o: ../lib/b.f90
gfortran -o lib.dir/b.o -c ../lib/b.f90  -M lib.dir
 
libmody.mod.proxy: lib.dir/b.o
 
lib.dir/b.o.requires: libmody.mod.proxy
 
lib.dir/requires: lib.dir/b.o.requires
 
 
lib.dir/clean:
rm lib.dir/a.o lib.dir/b.o lib.dir/libmylib.a
rm lib.dir/libmodx.mod lib.dir/libmody.mod
</pre>
 
build/prog.dir/build.make:
<pre>
prog.dir/all: prog.dir/prog
 
prog.dir/prog: ../../extLib/lib/libmyextlib.a
prog.dir/prog: prog.dir/main.o prog.dir/a.o ../../extLib/lib/libmyextlib.a
gfortran -o prog.dir/prog  prog.dir/main.o prog.dir/a.o  lib.dir/libmylib.a ../../extLib/lib/libmyextlib.a
 
prog.dir/a.o: ../prog/a.f90
gfortran -o prog.dir/a.o -c ../prog/a.f90 -M prog.dir
prog.dir/main.o:
prog.dir/main.o: ../prog/main.f90
gfortran -o prog.dir/main.o -c ../prog/main.f90 -I lib.dir -I prog.dir -I ../../extLib/include
 
 
localmod.mod.proxy: prog.dir/a.o
libmodx.mod.proxy: # dummy
externalmod.mod.proxy: # dummy
 
prog.dir/main.o.requires: localmod.mod.proxy libmodx.mod.proxy externalmod.mod.proxy
 
prog.dir/requires: prog.dir/main.o.requires
 
prog.dir/clean:
rm prog.dir/a.o prog.dir/main.o prog.dir/prog prog.dir/localmod.mod
</pre>
 
=== Rules like those that should be generated by CMake ===
 
build/Makefile:
<pre>
all: lib.dir/all prog.dir/all
 
lib.dir/all:
$(MAKE) -f lib.dir/build.make lib.dir/all
 
prog.dir/all: lib.dir/all
$(MAKE) -f prog.dir/build.make prog.dir/all
 
clean:
$(MAKE) -f prog.dir/build.make prog.dir/clean
$(MAKE) -f lib.dir/build.make lib.dir/clean
</pre>
 
build/lib.dir/build.make:
<pre>
lib.dir/all: lib.dir/mylib
 
lib.dir/mylib: lib.dir/libmylib.a
 
lib.dir/libmylib.a: lib.dir/a.o lib.dir/b.o
ar rc lib.dir/libmylib.a lib.dir/a.o lib.dir/b.o 
ranlib lib.dir/libmylib.a
 
lib.dir/a.o: ../lib/a.f90 lib.dir/libmody.mod
gfortran -o lib.dir/a.o -c ../lib/a.f90 -M lib.dir
touch lib.dir/libmodx.mod.stamp
lib.dir/b.o: ../lib/b.f90
gfortran -o lib.dir/b.o -c ../lib/b.f90  -M lib.dir
touch lib.dir/libmody.mod.stamp
 
lib.dir/libmodx.mod: lib.dir/a.o
lib.dir/libmody.mod: lib.dir/b.o
 
 
lib.dir/clean:
rm lib.dir/a.o lib.dir/b.o lib.dir/libmylib.a
rm lib.dir/libmodx.mod lib.dir/libmody.mod
</pre>
 
build/prog.dir/build.make:
<pre>
prog.dir/all: prog.dir/prog
 
prog.dir/prog: ../../extLib/lib/libmyextlib.a
prog.dir/prog: prog.dir/main.o prog.dir/a.o ../../extLib/lib/libmyextlib.a
gfortran -o prog.dir/prog  prog.dir/main.o prog.dir/a.o  lib.dir/libmylib.a ../../extLib/lib/libmyextlib.a
 
prog.dir/a.o: ../prog/a.f90
gfortran -o prog.dir/a.o -c ../prog/a.f90 -M prog.dir
prog.dir/localmod.mod: prog.dir/a.o
 
prog.dir/main.o: lib.dir/libmodx.mod.stamp
prog.dir/main.o: ../../extLib/include/externalmod.mod
prog.dir/main.o: ../prog/main.f90 prog.dir/localmod.mod
gfortran -o prog.dir/main.o -c ../prog/main.f90 -I lib.dir -I prog.dir -I ../../extLib/include
 
 
 
prog.dir/clean:
rm prog.dir/a.o prog.dir/main.o prog.dir/prog prog.dir/localmod.mod
</pre>
 
== Conclusion ==
 
=== What keeps CMake from doing it like could be done shown above? ===
 
For each target CMake parses the source files and writes the dependencies one by one.  This is ok for includes.  But doing it this way CMake cannot determine if a required module of source file ''a.f90'' is provided by a source file ''b.f90'' of the same target  or not.  This is IMHO the reason why the CMake developer droped a direct dependency of the source file to a required module.  As shown in the ''' rules like they are generated by current CMake''' sections an extra step called ''required'' was introduced.  This works out if one want just to build, but isn't enough for developer needs (because recompilation isn't done as expected).
 
=== What kind of changes have to be done to make it happen? ===
 
# Before the actually dependency tracking starts, CMake has to parse all fortran sources and to create a corresponding file i.e. ''mymodule.mod.stamp'' for ''mymodule''. This can be done source by source.  Now CMake is able to search the build tree for a module.
# Rather than doing it one by one, all sources of a target have to be parsed before starting to write dependencies.  This way CMake knows if a required modules is provided by itself or not.
# In case a required module ''mymod'' isn't provided by the same target
## search the build-tree for ''mymod.mod.stamp''
## if not found search it at the include paths
 
Module dependency tracking superseeds include dependency tracking but
: '''Note:''' IMHO the code responsible for C/C++/Java dependency generation shouldn't be touched, since speed is an imporant advantage of CMake for developers!
 
=== Notes from Brad ===
 
The "provides", "requires", and "proxy" stuff is necessary for correct minimal rebuilds even within a single target.
<!-- -->
: '''Maik''': AFAI understand its necessary since there are no direct dependencies of target sources to the mod files, right?
 
However it has not been correctly implemented in the latest Makefile generator. 
<!-- -->
: '''Maik''': I know cmake since 2.4.2, but AFAIK former versions generated one big Makefile, right?
<!-- -->
The idea is that after recompiling a source file that provides a module, there should be a copy-if-different step to update the mod.stamp file. 
<!-- -->
: '''Maik''': And this stamp is useless since the last Makefile generator was introduced.
: '''Brad''': Like I said it is not correctly implemented.  The .o files need to depend on the .stamp files.
<!-- -->
Then there must be a <b>recursive</b> call to make for each source that requires a module to evaluate the file-level dependencies with the updated stamp file.
 
I do not think we can search the whole build tree for mod.stamp files.  The trees can be way to big and that would take forever.
<!-- -->
: '''Maik''': ...(1)
: '''Brad''': There will have to be some kind of persistent scanning results for (1) to work.  We cannot scan the whole tree every time CMake runs.
<!-- -->
Also there could be leftover stamp files from modules that were once available and are not anymore or were moved. 
<!-- -->
: '''Maik''': good point! I didn't consider this.
<!-- -->
Also, creating inter-target dependencies cannot be done during the dependency scanning stage, so we cannot create such dependencies based on information implicit in the source files.  However, if one target is importing the modules from another target, it must link to that target in order to get the implementation.  That linking should create the explicit inter-target dependency that is necessary. 
<!-- -->
: '''Maik''': Yes, that is I have in mind.  We only have to search for implicit dependencies where explicit ones are given by the user.  I think this solves your concern (1) too.  Additionally we only have to search in fortran projects.
<!-- -->
This just leaves creating the dependencies on the mod.stamp files for modules provided in other targets.  I'm still thinking about this one.
 
==== Maik ====
 
I will delete my notes on your notes ;) as soon as I'm sure I didn't get you wrong.
 
The stamp file deletion problem is indeed tricky. I think the extra step which creates them should look for existing to delete them before writing new.  The question is how can CMake determine if a fortran-target source got rid of a module to trigger this extra step.  I doubt this is possible, so user interaction is required (rerun-CMake).
 
BTW: CMake only has to create a stamp file if the source file which contains the corresponding module belongs to a lib-target.
 
==== Brad ====
 
I've updated http://www.cmake.org/Bug/view.php?id=5809 with a note at "10-12-07 09:53".  A partial fix for file-level dependencies with module timestamps has been implemented.  Rebuilding should now work correctly for all object files whose required modules are provided in the same directory (even if in different targets).

Latest revision as of 15:41, 30 April 2018


The CMake community Wiki has moved to the Kitware GitLab Instance.

This page has moved here.