[CMake] ANN: UseOcaml.cmake

Judicaël Bedouet j.bedouet at infonie.fr
Mon Jan 25 18:28:37 EST 2010


Hi,

I think I have found the problem. It's in the structure of the Makefile
generated by CMake. For each target, make is called on a sub Makefile and
this is problematic for OCaml. I have reproduced the problem with an
handwritten Makefile (see attachment).

Try make -j all2 and make -j all1. The first command will always work
whereas the second, with recursive calls to make, will fail sometimes.

Recursive Makefiles with OCaml seem to be incompatible with parallel make.
In OCaml, each object depends on others. If two targets depend on io.cmx and
if make is launched in parallel on these two targets, the file io.cmx will
be built twice. And if the first make command builds the object whereas the
second tries to link the executable at the same time, the link command will
fail because the object is being corrupted.

I think it's only a problem for OCaml. In C or C++, objects only depend on
sources, and not on objects previously compiled.

Do you agree with this explanation ?

I see one solution : add a macro to create an only target with all OCaml
libraries and executables, but it's not perfect.

Bye,

J. Bedouet



2010/1/22 Keyan <ml at pulsschlag.net>

> hi,
>
> i didnt have much time lately to look deeply into the scripts. at first i
> thought that one has to add some add_dependencies, which was my first
> strategy, before i wrote useocaml.cmake. but then i saw, that i removed
> them, because i got the DEPENDS right. so my guess was wrong.
>
> what you write seems plausible. i got the messages with your script when i
> compiled with make -j 2 while my computer was busy. for some reason i didnt
> get them with my scripts yet, which could just be by chance.
>
> i do not know how to proceed from here on.
>
> anyone got an idea?
>
> cheers,
> keyan
>
>
>
> On 22 Jan 2010, at 02:16, Judicaël Bedouet wrote:
>
> > Hello,
> >
> > I try to compare Makefiles generated by your scripts and mine. I don't
> see any big differences, except the copy rules.
> >
> > While trying your scripts, I also reproduced a similar bug.
> >
> > Generating file_node.cmx
> > Generating message_node.cmi
> > Generating file_node.cmi
> > File "/Users/judi/src/ocaml/cmake/
> > code-analysis/build/file_node.ml", line 1, characters 0-1:
> > Error: Corrupted compiled interface
> > file_node.cmi
> > make[2]: *** [file_node.cmx] Error 2
> >
> > Try with
> > while true; do make clean; make -j; done
> > and you will reproduce it.
> >
> > I think dependencies in our scripts are good (or almost good). The reason
> may be elsewhere.
> >
> >
> > I don't see a solution yet but I have found a promising lead. When two
> targets depends on the same object, make tries to build the object twice.
> >
> > If make builds the same object twice at the same time, the command 'cmake
> -E copy_if_different' may be called at the same time and one of the cmake
> command fails because it can't open the file. I suppose it's the reason of
> >
> > > Error copying file (if different) from
> "[...]/code-analysis-build/Dependencies/message_node.ml.dep.cmake.tmp"
> > > to "[...]/code-analysis-build/Dependencies/message_node.ml.dep.cmake".
> >
> > In your code, the loc and messages executables depends both on io.cmo.
> Sometimes, it tries to link loc and build io.cmo (probably for messages) at
> the same time. I suppose it's the reason of
> >
> >
> > > Building OCaml object io.cmo
> > > Building OCaml object files.cmo
> > > [  0%] Linking OCaml executable loc
> > > File "_none_", line 1, characters 0-1:
> > > Error: The file /Users/judi/src/ocaml/cmake/code-analysis/build/io.cmo
> is not a bytecode object file
> >
> >
> > I add a third target which depends on io.cmo (create a common library and
> make loc and messages depend on this library). It appears that make tries to
> build the object io.cmo three times and build fails are almost systematic.
> >
> >
> > If you keep only one target, there is never no problems.
> >
> >
> > An idea ? Too late in the night for me :-(
> >
> >
> > Bye,
> >
> > J. Bedouet
> >
> >
> >
> >
> > 2010/1/20 Keyan <ml at pulsschlag.net>
> > hi,
> >
> > i started to work on your scripts. very nice work!
> >
> > unfortunately, i couldnt find the problem, and i cant reproduce it with
> my scripts. there must be a difference somewhere.
> >
> > cheers,
> > keyan
> >
> >
> >
> > On 19 Jan 2010, at 00:30, Judicaël Bedouet wrote:
> >
> > > Hi,
> > >
> > > Effectively, I have reproduced your two bugs by using make -j. It's not
> systematic (once in ten).
> > >
> > > >> Error: The file /Users/zahedi/Downloads/code-analysis-build/io.cmo
> is not a bytecode object file
> > > When this error occurs, it also appears that the object is being built.
> And if I rerun make, linking is correct, without rebuilding io.cmo.
> > >
> > > Building OCaml object io.cmo
> > > Building OCaml object files.cmo
> > > [  0%] Linking OCaml executable loc
> > > File "_none_", line 1, characters 0-1:
> > > Error: The file /Users/judi/src/ocaml/cmake/code-analysis/build/io.cmo
> is not a bytecode object file
> > >
> > > Dependencies in the generated Makefile seem to be correct :
> > > loc: file_node.cmo
> > > loc: files.cmo
> > > loc: io.cmo
> > > loc: file_statistics.cmo
> > > loc: message_node.cmo
> > > loc: loc.cmo
> > >
> > > Is OCaml parallel make safe ? I'm not sure... I readd CMake address :
> maybe someone has an idea...
> > >
> > > To speed up compilation, I could look for ocamlc.opt and if not found
> ocamlc. Bytecode compilations should be faster.
> > >
> > > J. Bedouet
> > >
> > >
> > >
> > > 2010/1/18 Keyan <ml at pulsschlag.net>
> > > hi,
> > >
> > > i am just trying to reproduce the message i posted. i cant, but i got
> something else:
> > >
> > > Error: The file /Users/zahedi/Downloads/code-analysis-build/io.cmo is
> not a bytecode object file
> > >
> > > i use make -j 2 (at least 2), and my guess is that it has to do with
> parallel compilation. could that be the case?
> > >
> > > cheers,
> > > keyan
> > >
> > > On 18 Jan 2010, at 18:01, Judicaël Bedouet wrote:
> > >
> > > > Hi,
> > > >
> > > > >> first of all, wow, good job! very nice scripts, much more
> sophisticated then mine :)
> > > > Thanks :-)
> > > >
> > > > >> I only had one error when first trying: Error copying file (if
> different) from "[...]/code-analysis-build/Dependencies
> > > > >> /message_node.ml.dep.cmake.tmp" to
> "[...]/code-analysis-build/Dependencies/message_node.ml.dep.cmake".
> > > > Maybe the Dependencies directory is not created. It is done at line
> 32 of OCamlDep, though. Let me know if you find something.
> > > >
> > > > I have tested my scripts with OCaml 3.10 and I have found that
> ocamlfind is officially distributed with OCaml since 3.11. Maybe, I should
> put ocamlfind stuff in a FindOCamlFind script...
> > > >
> > > > J. Bedouet
> > > >
> > > >
> > > >
> > > > 2010/1/18 Keyan <ml at pulsschlag.net>
> > > > Hi,
> > > >
> > > > i am sorry that i could not reply any sooner.
> > > >
> > > > first of all, wow, good job! very nice scripts, much more
> sophisticated then mine :)
> > > >
> > > > i only had one error when first trying:
> > > >
> > > > Error copying file (if different) from
> "[...]/code-analysis-build/Dependencies/message_node.ml.dep.cmake.tmp" to
> "[...]/code-analysis-build/Dependencies/message_node.ml.dep.cmake".
> > > >
> > > > i will try to solve that myself. this way i know if i understand your
> scripts.
> > > >
> > > > cheers,
> > > > keyan
> > > >
> > > > On 16 Jan 2010, at 14:51, Judicaël Bedouet wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Here are my OCaml scripts, with your OCaml code (it helped me to
> find a bug about a source file included in two targets).
> > > > >
> > > > > I have kept your idea about an only macro to compile and link,
> reused your macro ocaml_parse_arguments (I should use for my other macros
> :-) ) and added a macro find_ocaml_package to find ocaml packages with
> ocamlfind.
> > > > >
> > > > > Tell me what you think,
> > > > >
> > > > > J. Bedouet
> > > > >
> > > > >
> > > > >
> > > > > 2010/1/15 Keyan <ml at pulsschlag.net>
> > > > >
> > > > > hi,
> > > > >
> > > > > > Indeed, it is much more sophisticated, so it may be more bugged
> ;-)
> > > > >
> > > > > i can help debugging :)
> > > > >
> > > > > > >> shell i send you my scripts to take a look?
> > > > > > It's not necessary. I have downloaded the refactoring branch of
> yars. Thanks.
> > > > >
> > > > > oh, ok, then you should have the newest version (last yars revision
> is 513).
> > > > >
> > > > > > I comment my OCaml scripts and I send you, so that you could also
> have a look.
> > > > >
> > > > > great.
> > > > >
> > > > > > I have took a look at your UseOCaml script. I can't see why you
> copy ML files into the current binary dir before compiling them. Is there a
> reason ?
> > > > >
> > > > > no. i copied most of the stuff from the UseLATEX.cmake. it was done
> so there, and at the time i wrote the script, i thought its a good idea.
> don't ask me why :)
> > > > >
> > > > > > Keep in touch,
> > > > >
> > > > > let me know when i can jump in and help. looking forward.
> > > > >
> > > > > cheers,
> > > > > keyan
> > > > >
> > > > > > J. Bedouet
> > > > > >
> > > > > >
> > > > > > On Fri, Jan 15, 2010 at 8:26 AM, Keyan <ml at pulsschlag.net>
> wrote:
> > > > > > I totally agree.
> > > > > >
> > > > > > Judicaël: how should we proceed? my first impression is that your
> usecaml.cmake is much more sophisticated then mine :) same for the
> findocaml.cmake scripts, as mine up to now does nothing more than to call a
> bunch of find_program commands.
> > > > > >
> > > > > > shell i send you my scripts to take a look?
> > > > > >
> > > > > > i have no problem with merging my stuff into your scripts,
> keeping you as main author, if there is anything useful for you.
> > > > > >
> > > > > > cheers,
> > > > > > keyan
> > > > > >
> > > > > >
> > > > > > On 15 Jan 2010, at 02:39, Philip Lowman wrote:
> > > > > >
> > > > > > > On Thu, Jan 14, 2010 at 5:11 AM, Keyan <ml at pulsschlag.net>
> wrote:
> > > > > > >> hi,
> > > > > > >>
> > > > > > >> ups ... i thought i replied to the list. thanks for the link.
> i will adapt my find-script to the requirements and maintain the package.
> > > > > > >
> > > > > > > You should work with Judicaël to combine the best of both of
> your
> > > > > > > works together into one FindOCaml and UseOCaml script.  There
> is no
> > > > > > > sense having two of them out there.
> > > > > > >
> > > > > > > --
> > > > > > > Philip Lowman
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> >
> >
> >
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.cmake.org/pipermail/cmake/attachments/20100126/0cb40909/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Makefile
Type: application/octet-stream
Size: 852 bytes
Desc: not available
URL: <http://www.cmake.org/pipermail/cmake/attachments/20100126/0cb40909/attachment-0001.obj>


More information about the CMake mailing list