[CMake] Contribute Find-module to CMake vs Config-file to upstream

[Mateusz Loskot]

On 22 May 2018 at 13:36,  <rleigh at codelibre.net> wrote:
> On 2018-05-21 18:39, Mateusz Loskot wrote:
>>
>> I've been recently trying to update/add Find-modules to CMake:
>> updated FindJPEG, proposed FindODBC and most recently FindLZ4.
> […]
>>
>> The FindLZ4 discussion basically ended with suggestion from Brad that,
>> instead of adding Find-module to CMake, I should approach LZ4 project
>> and add Config-file to the library itself.
>> Then I argued taht Config-files are more complex and I don't feel like
>> asking projects, which I don't maintain myself, to accept the extra
>> complexity.
>> Finally, Brad replied that CMake doesn't need LZ4 itself and yet for some
>> reason is expected to know all about it: versions, headers, libraries,
>> etc.
>> and that it's not scalable to teach CMake about every project in the
>> world.
>
>
> It really doesn't scale.  I've contributed and provided ongoing fixes for a
> few Find modules out of practical necessity, for projects which are
> effectively incapable of adding a Config-file, either because the project
> was stagnant with no new releases (Xerces, Xalan), or where the project was
> so complex that it was effectively impossible to add (Boost).

It does not matter if it scales.
If there is enough interest within the community, Find-modules should be
allowed and maintained. If they are not, they will stagnate.
That will be the community choice.
Still, it will not hurt CMake principles which already does not care.

> However, that work should have been done by the upstream.
> The ongoing maintenance cost
> has been moved from the upstream development, where it belongs, to a
> volunteer (me).  It's work which I do because it provides value to me and my
> employer so I can use CMake with the projects I work on, but is otherwise
> unjustifiable.

Regarding FindBoost.cmake, being a Boost user, as well as
developer and maintainer of Boost libraries, I already know it is PITA.
I, however, have learned FindBoost.cmake is always step behind
and I have learned how to live with that.
There have been discussions about how Boost could take ove FindBoost.cmake
or provide Config-files, but it always ended with discussions about
(too) clever automatic solutions, Boost.Build integrations, bla bla bla
that literally no one is knowledgable enough or interested to maintain
in long term
- should read as "it's going to implode sooner or later anyway".

To me, FindBoost.cmake works in majority of cases (perhaps not quite
for 1.67 ;))
I know enough CMake scripting to be able to attempt to fix stuff.
Better is enemy of good. I prefer good as good enough to get things done.

Finally, Find-modules are just "guesser", they do not promise marksman-like
operating to find every variant of every version of every library.
Find-modules, however, should be there for most, even if lacking
they can serve as useful basis to develop improved Find-modules,
perhaps contribute it back.
If there is not CMake-provided Find-module, then there is cultivation of
chaos of mostly badly written Find-modules scattered caross GitHub
(thanks the spirits GitHub provides usable search engine).


> You simply can't expect every Find module to keep pace with
> upstream development without a small army of volunteers to keep them all up
> to date.

The point is, I do not expect that.
I have mentioned multiple times, Find-modules should be allowed to
become outdated,
awaiting for intersted volunteers to deliver necessary updates.
But, Find-modules should be there regardless.

> If they are provided with the upstream releases, they are de facto
> always up to date because they are directly matched to the library.

IMHO, that is a false expectation or perception.

> Note that for several of these projects (e.g. Xerces, libtiff), I have
> actually provided upstream CMake build systems which install Config-files,
> so in the future the Find modules can be deprecated and users can move to
> the Config-file as these releases make their way into distributions.


Again, my point is, there is no need for deprecating Find-modules.
CMake can automatically prefer Conig-files if present.
CMake users can conciously prefer one or the other.
Point being, there is choice and the choice is left to both sides,
users as well as maintainers, including Find-modules maintainers.

> That's semi-altruistic.  I've spent a lot of effort up front to make CMake
> infrastructure in these projects maintainable for the long term, and
> something which can be delegated to others for ongoing maintenance.

I do realise the hassle may be huge.
Why not simply shift the way we think about Find-modules
- let them in, without deprecating, and let them remain as fall-back
(regardless if
up-to-date, working or not).

>> This opinion does not surprise me and I understand the rationale.
>> I argue that CMake does need to know about every project in the world,
>> or every project that CMake users wish to use, directly or indirectly.
>> Without the knowledge CMake would simply be useless.
>
>
> Not useless, that's going a bit far.  It's requiring a little bit of extra
> work to be functional.  For libraries without Find modules, you can usually
> write your own in a couple of dozen lines.

Yet another point being, I do not want to write my own, maintain
'cmake' subfolder
in numerous projects of mine, I do not want that.
I buy in to CMake to get it covered by CMake
(even if it means I write a Find-module myself and contribute it to CMake).


> At that point your options are:
>
> - to upstream it as a Config-file template to the upstream developers
> - to maintain it independently in your project
> - to upstream it to cmake.git

I aim to eliminate the 2nd option, but sometimes the 3rd option is not feasible
due to CMake bouncing the responsibility to 1st option.

>> If a library is strictly GNU Autoconf, I'm on lost position trying to
>> convince
>> maitnainers to accept CMake stuff and even if they do, they won't be
>> willing to
>> maintain it - broken Config-files may be deployed with new packages.
>> That puts users of CMake and such library on lost position:
>> CMake says no to Find-module, the library does not care either.
>
>
> If it's using Autoconf, then it can still provide and install CMake
> configuration files.  It can use @var@ substitutions in the Config-file.in
> template and then install it.  Just like for pkg-config templates.

Unless, a religious unwillingless plays an important role.

> Individual projects may have their own policies, and some might not want to
> have CMake support.  In general though, if you were to provide a tested
> patch that creates the Config-file in AC_OUTPUT and then install as e.g.
> cmake_pkgconfig_DATA in Makefile.am, add a distcheck hook and/or CI logic to
> test it, and most projects are in my experience willing to accept the patch.
> The only way to find out is to create and submit the patch, and have a
> conversation with them about the cost/benefit of including it.  Some might
> be worried about the maintenance burden, so explain the value of it and
> maybe commit to maintaining it as required.  Since you'd likely be
> maintaining it /anyway/, it's not a big task.

As I have explained in my initial post, I may not always be
intersested in maintaining
CMake support in N different locations/projects/repositories (for
variety of reasons).
I, however, may be happy to maintain it in the single CMake upstream repository.

>> As CMake user I don't want to be left to the discretion of package
>> maintainers.
>> Packages may ignore CMake Config-file existence.
>> Packages of libraries which provide CMake and alternative build
>> configurations
>> may not deploy Config-files or Find-modules.
>>
>> IMHO, CMake should encourage contributions of new Find-modules.
>
>
> As with all things, in my opinion this really comes down to these questions:
> "who is going to do the work", and "who is paying for their time".
> Contribution also has implied ongoing maintenance obligations.

That is excatly what I'm arguing against.

I have submitted number of m4 modules to GNU Autoconf Archive
and nobody objected (unless a module was technically ill-formed)
or asked me if I'm going to maintain those modules.
In fact, I have not done any maintenance for long time, but I have received
numerous patches from users (which I submitted back to the Archive, of course).


> If Find modules are contributed and then not maintained, they will become broken
> over time, and who will be expected to keep them updated?  The CMake core
> maintainers?  The submitter?  Users who need it?
> If it's upstreamed, it's much less of a liability.

I will repeat myself once again: CMake aleady does not care or at least can not
promise a Find-module is up to date. So, there is no harm of a collection of
*official* CMake modules installed aside of CMake is left to be
maintained by community.

> For the stuff I've contributed, it's mostly been
> done on my paid work time because it has value for the work projects,
> enabling easy building and testing on a large range of platforms for both
> our developers and end users.  But when I move onto new projects or a new
> job, I'll find it difficult to justify doing this in my spare time, as much
> as I might like to.  If you take a look at the bug list for e.g. FindBoost,
> there are a lot of open issues, but it's so huge and complex that I don't
> have the resources (different systems) to address all of the defects--that
> requires contributions from the people who can actually test these systems I
> can't.  The huge complexity here squarely belongs upstream.  FindBoost is
> one of the more complex modules, and I can fully sympathise with the desire
> to avoid more of these maintenance nightmares.  It's already a large burden
> upon both people like me, and I suspect also the CMake core maintainers in
> creating, testing and reviewing changes for every Boost release and for
> every platform it's expected to run on.  Multiply that by the total number
> of Find modules (approximately 150 at present).


A module can naturally become outdated. If there is nobody submitting patches,
there should be no problem and CMake should not suffer at all.
It already does not promise up to date Find-modules.
But, IMHO, existence of FindX.cmake increase chances a contributor
will arrive with updates.
If there is no FindX.cmake, there is nothing to update, because there
is nothing to maintain.


> One thing I do think could be improved is the behaviour of find_package and
> MODULE|NO_MODULE|CONFIG.  In order to allow more transparent migration from
> Find modules to Config-files it would be nice if in the absence of any of
> these flags, it would firstly try the Config-file and if it was missing,
> then fall back to a Find module.


I agree. That is should be a must-have feature.

I would even dare to judge that lack of this feature is one of reasons
CMake maintainers dislike Find-modules :-)


Best regards,
-- 
Mateusz Loskot, http://mateusz.loskot.net