[CMake] [cmake-developers] Using CMake as a package manager vs using a dedicated package management tool (like Conan)

[Timothy Wrona]

Correction:

*I haven't tried this yet, but I am hoping it will work well* - Pull Put my
sub-projects (my own custom libraries) into their own independent git repos
and pull them into my main project using "FetchContent". Then when I run
"FetchContent" it will checkout the sub-projects and I will have all of the
source code available. I am hoping if I do this any changes I make to the
sub-projects can easily be committed and pushed back to their own
independent repositories.

On Tue, Feb 19, 2019 at 10:33 AM Timothy Wrona <tjwrona1992 at gmail.com>
wrote:

> Hi Craig,
>
> Thank you for the detailed description!
>
> To answer some of your questions:
>
>    - This project will not be incorporated into a Linux distribution,
>    however I would like to keep it cross platform and it should work on
>    Windows, Mac, and Linux.
>    - All of the pieces of the project that I am writing myself are using
>    CMake, but I do have some dependencies on external libs such as
>    "googletest" and "boost". Conan does seem to make using these external
>    dependencies very simple - especially since I am using MinGW to do my
>    compilation and "googletest" doesn't seem to compile out of the box on
>    MinGW without passing specific flags to the compiler. With Conan I get a
>    pre-compiled binary so it just works out of the box.
>    - At the current time, this project does not have many dependencies,
>    although it will likely grow quite large. I don't believe the external 3rd
>    party dependencies will need to update frequently, but all of the libraries
>    I am writing for the project will likely change quite a bit throughout
>    working on the project. I would also like to be able to compile these
>    libraries independently for reuse in other projects.
>    - I intend to support as many platforms/compilers as possible, I am
>    currently using Qt as my dev environment which allows me to
>    install/configure multiple compilers and try builds with each one - this
>    way I can at least do a build with both MinGW and msvc very quickly.
>    - I would like to support tools like sanitizers and code refactoring
>    tools at least within my own libraries (not necessarily with any of the 3rd
>    party libs)
>    - I would like past releases to be repeatable/rebuildable as much as
>    possible, although when the compiler is upgraded it is understandable that
>    past versions may have issues. I don't have much understanding of CI
>    systems at this point and it is something I have been trying to become more
>    familiar with. I'd like to avoid having multiple versions of the same
>    dependency, although I don't think having two versions of "googletest" for
>    two separate sub-projects that don't depend on each other would cause any
>    issues.
>       - I imagine Conan would make past releases more repeatable since
>       you can fetch binary packages instead of needing to rebuild from source and
>       package versions are always explicit.
>
> I am currently the only developer working on the project, but I would
> still like to find the most efficient method of managing packages for my
> particular situation since it will likely save me a lot of pain down the
> road if I get it right up-front. This project is in the situation where I
> have a set of different sub-projects (libraries) that all need to be able
> to be compiled independently for re-use, but I have a main project that
> will use them all. All of these projects have their own dependencies (some
> of them 3rd party, some of them written by me). The 3rd party libs will
> likely rarely change, but the ones written by me may change quite
> frequently.
>
> This is an approach I was thinking about taking:
>
>    - *So far I have tried this and it is working well* - Manage 3rd party
>    libs with Conan, since I don't need to see the source and it's quite
>    convenient to not need to recompile them this seems to work pretty well.
>    Using "FetchContent" on 3rd party libs and then attempting to compile them
>    yourself can sometimes be tricky (for example "googletest" requires special
>    compiler flags to be compiled with MinGW.) It also ensures that if two
>    projects ask for the same version of a dependency I get only one copy even
>    if the projects are built entirely independent of each other.
>    - *I haven't tried this yet, but I am hoping it will work well* - Pull
>    my sub-projects (my own custom libraries) into their own independent git
>    repos and pull them into my main project using "FetchContent". Then when I
>    run "FetchContent" it will checkout the sub-projects and I will have all of
>    the source code available. I am hoping if I do this any changes I make to
>    the sub-projects can easily be committed and pushed back to their own
>    independent repositories.
>    - The alternative to this would be to put my own sub-projects into
>       their own Conan packages and use Conan to get them from the main project.
>       But I am thinking since they will change frequently, "FetchContent" may be
>       a better fit for this scenario.
>       - Maybe when the sub-project libraries reach a mature and stable
>       release they should be packaged into Conan and fetched in the main project
>       from Conan at that point?
>
> Let me know what you think! :)
>
>
> On Tue, Feb 19, 2019 at 5:56 AM Craig Scott <craig.scott at crascit.com>
> wrote:
>
>>
>>
>> On Tue, Feb 19, 2019 at 12:46 PM Timothy Wrona <tjwrona1992 at gmail.com>
>> wrote:
>>
>>> I have been working on a new C++ project and I am trying to decide
>>> whether I should use CMake as my package management system or if I should
>>> use a dedicated package management tool such as Conan.
>>>
>>> For more information on Conan see: https://conan.io/
>>>
>>> I am trying to understand the main difference between using Conan to
>>> manage dependencies vs using CMakes "FetchContent" module. Are there any
>>> compelling reasons to prefer something like Conan?
>>>
>>
>> Excellent question, one that I think deserves a more detailed answer than
>> I can provide here, but I'll try to hit the main points as I see them.
>>
>> Personally, I think there is no "right answer" or "one size fits all"
>> when it comes to package management for a CMake project. What works well
>> for one situation, person or project may not be as convenient or suitable
>> for another. There are competing needs and views, some of which are
>> personal preferences, others are hard requirements from things like OS
>> distribution policies for packaging. Even just the maturity of a project
>> can have a big influence on how developers may prefer to handle its
>> dependencies and handle it as a dependency of other projects.
>>
>> The key thing for me is that the developer should ideally have choices
>> when it comes to this area. If a project hard-codes that its dependencies
>> must come from a particular provider (whether that be Conan, Hunter, vcpkg
>> or some other system), this might not be compatible with what the
>> developer's situation allows. You would need to weigh up whether it makes
>> sense to lock the developer into a particular package manager if they want
>> to use your project or not. An inappropriate choice here can mean lower
>> adoption of the project as some may reject it for consideration based on
>> this point alone.
>>
>> If instead a project relies only on find_package() to find its
>> dependencies, then it is up to the developer to ensure they are all
>> available. This could be done using whatever package manager the developer
>> finds convenient, or they could build the dependency projects from source
>> individually or they might set up a superbuild parent project that builds
>> the dependencies in the required order and makes dependees available to
>> dependers. This gives good flexibility at the cost of more responsibility
>> on the developer than perhaps some would want (again, it will be highly
>> situation-dependent).
>>
>> A drawback with find_package() is that it assumes you actually have a
>> packageable project. For a variety of reasons, this may not be the case.
>> Consider a large, complex project in its early stages and where multiple
>> teams are working on different subprojects which all get combined into some
>> larger whole. Each of the subprojects may need to be able to build on their
>> own with their own smaller subset of dependencies, but they also need to be
>> able to be incorporated into a larger build (think of different teams
>> working on core toolkits, rendering engines, different algorithm
>> strategies, multiple GUI applications, backend components, etc). No-one may
>> know yet how it should get packaged up and everyone might be focused on
>> just getting a minimal viable prototype up and running as a technical
>> demonstrator. For a case like that, neither find_package() nor a package
>> manager really fits the workflow. In this situation though, FetchContent is
>> a perfect fit, since it doesn't require any packaging to already be in
>> place, it needs no external tools other than CMake and it gives each
>> project precise control over its dependencies down to the individual commit
>> to bring in for each one.
>>
>> With the above in mind, perhaps the following few questions may be
>> helpful in clarifying what your constraints are and maybe steering you more
>> toward one way or the other:
>>
>>    - Will the project be incorporated into a Linux distribution at some
>>    point (not just be installed on Linux, but be part of the actual Linux
>>    distribution as provided by its own native package manager)? If so, I would
>>    expect this would pretty much eliminate using any package manager and
>>    instead require that you use find_package() to find all dependencies.
>>    - Are any of the dependencies of the project using a build system
>>    other than CMake? If so, they tend to take a bit more work to incorporate
>>    into a build via ExternalProject or FetchContent. If you can assume the
>>    developer provides those somehow, bringing them in by find_package() shifts
>>    responsibility for building them from the project to the developer (or
>>    whatever package manager they choose to use). This might be anywhere from
>>    entirely appropriate to entirely problematic depending on who your intended
>>    audience is.
>>    - How many dependencies does the project have and what is the
>>    maturity of each one? Will the project need to update any of those
>>    dependencies often (are they also being actively developed, do you need to
>>    follow recent work in them)? What will be the impact on developers working
>>    on the project when these dependencies need to be updated (how easy is it
>>    for them to update, what assumptions are you making about their development
>>    environment and tools)?
>>    - What breadth of platforms, compilers and CMake generators do you
>>    want to support? The bigger this set, the more it may drive you towards
>>    building dependencies from source rather than relying on having pre-built
>>    packages available to you.
>>    - Do you want developers to be able to use tools like sanitizers,
>>    perform code refactoring across projects or have source code visibility
>>    into dependencies within their IDE tools? FetchContent supports all of
>>    these requirements quite naturally, but doing so for the other methods it
>>    may be more difficult.
>>    - How will you manage repeatability of building past releases? Will
>>    you be able to build a year old release again if you update a build slave?
>>    In particular, what assumptions are you making about a CI system's build
>>    slaves and what dependencies they have installed (can you have multiple
>>    versions of any given dependency available at once, whether that be at a
>>    known path or via whatever package manager(s) you choose to use)?
>>
>>
>> On a side note, this is an area I'm increasingly thinking about these
>> days (I'm the author of the FetchContent module). I'm interested in hearing
>> peoples' views on dependency management and building an understanding of
>> what works for people and what doesn't. If you want to send feedback my
>> way, do feel free to get in touch.
>>
>> --
>> Craig Scott
>> Melbourne, Australia
>> https://crascit.com
>>
>> Get the hand-book for every CMake user: Professional CMake: A Practical
>> Guide <https://crascit.com/professional-cmake/>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://cmake.org/pipermail/cmake/attachments/20190219/51aa95c2/attachment-0001.html>