[CMake] Dependencies with static libraries

[Stephan Menzel]

On Thu, Jun 28, 2018 at 4:07 PM Stephan Menzel <stephan.menzel at gmail.com>
wrote:

> I am observing a strange problem regarding dependencies to static libs in
> executables. I am not entirely sure but it seems this started since I
> upgraded to the 3.11 line. I am using Windows and MSVC14.
>
> Basically, my code base contains a number of static libraries that are
> linked together to a few executables. Very basic and down to earth. Like
> this:
>
> set(MY_SERVER_SRC
> ServerSource1.cpp
> ServerSource2.cpp
> )
>
> set(MY_SERVER_HDR
> ServerConfig.hpp
> Header1.hpp
> Header2.hpp
> )
>
> add_library(my_server STATIC ${MY_SERVER_SRC} ${MY_SERVER_HDR})
> target_link_libraries(my_server
> some_tools
> some_base
> ${Boost_LIBRARIES}
> ${OPENSSL_LIBRARIES}
> )
> add_executable(serverd main.cpp)
> target_link_libraries(serverd
> my_server
> some_tools
> some_base
>
> ${Boost_LIBRARIES}
> ${OPENSSL_LIBRARIES}
> )
>
> So a few big libs with the beef in them and then a small main.cpp
> containing executable. Nothing fancy.
>
> What gives me trouble now is that dependencies within that seem broken. In
> above example, when I edit ServerSource1.cpp and then hit "Build", I see
> ServerSource1.o being compiled and the library my_server being linked. Like
> expected. Problem is, serverd does not get linked. Or at least I see no
> hint of it being linked (In MSVC's output window).
> So, my question is: What could cause this? I there anything 3.11.x does
> differently which causes those dependecies to break? And how can I fix this?
>
> This is quickly turning into a major problem for us here as it means that
> we cannot reliably build anymore because whenever we change something and
> hit build we can't be sure we get a linked executable anymore unless we do
> a complete rebuild, which takes a long time (In fact, I wrote this message
> waiting for one).
>
>
I have a little update here in case anyone experiences the same. Since I
first noticed it when I posted here the problem was degrading. Back then I
thought I couldn't track it down because it wasn't always happening. What
led me to believe it was reproducible was the lack of any kind of output in
the console.

Unfortunately since then the problem has turned from a major nuisance into
a much bigger issue because right now it actually is always reproducible.
It never works anymore.

The only way to bring VS to re-link my executable is to delete the old one.
Any change in the lib 'my_server' leads to the lib being compiled and
linked fine but the server executable never is.

I have checked this many times using timestamps. Now I usually see the lib
with a 'right now' timestamp and the executable with a much older one.
Several other developers here in the team experience the same thing so I
don't think it is my computer or the installation or anything.
I have also tried updating to 3.12.1 and the issue is still there.

As for reasons I am still in the dark. My money would be on some kind of
race condition. When the problem gradually got worse I also occasionally
saw a linker error saying the "program database is already in use". Waiting
a few seconds and then clicking "Build Solution" again generally solved
that but still I suppose this should never happen unless the binary is
running or otherwise in use.

Also since then we gradually added rather template heavy code to that
static lib which (in Debug mode) is now 250M in size. My guess is that some
processes are still working on the lib when the linker tries to build the
executable out of it. The linker fails, lack of console output hides that
from me and with the lib growing this got more and more likely over time
and is now always the case.

I know, it's a long shot. Counter evidence against this hypothesis is the
fact that in Release mode the problem occurs just as much and there the
data is much smaller, only about 50M, with no pdbs.

Meanwhile I have also double checked all kinds of flags or defines we use
and did not find anything out of the ordinary. I have also verified the
dependencies as they are shown in the visual studio project properties.
They look very much like I expect them to look.

I am doubling down efforts into investigating this now as it has cost us
dearly the last couple of weeks as new developers stumbled into it without
knowing about it. Will post results here. Of course I also appreciate new
ideas about possible reasons.

Cheers,
Stephan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://cmake.org/pipermail/cmake/attachments/20180816/c17e44ac/attachment-0001.html>