[CMake] providing library information, what's the cmake way

[Johannes Zarl]

On 12/01/2010 at 16:06, Michael Hertling <mhertling at online.de> wrote: 
>> FIND_PACKAGE(XXX COMPONENTS YYY)
>> ...
>> ADD_SUBDIRECTORY(subdir)
>> ...
>> TARGET_LINK_LIBRARIES(AAA ${XXX_LIBRARIES})
>> TARGET_LINK_LIBRARIES(BBB ${XXX_LIBRARIES} ${XXX_YYY_LIBRARIES})
>>
>> In subdir/CMakeLists.txt:
>>
>> FIND_PACKAGE(XXX COMPONENTS ZZZ)
>> ...
>> TARGET_LINK_LIBRARIES(subBBB ${XXX_LIBRARIES} ${XXX_ZZZ_LIBRARIES})
>>
>> As I mentioned above, I would expect XXX_LIBRARIES to contain only the
>> base library, and find_package calls to act accumulatively.
> 
> If I understand correctly, you'd like to have a set of variables like
> XXX_*_{LIBRARIES,INCLUDE_DIRS,...} for each component instead of one
> comprehensive set XXX_{LIBRARIES,INCLUDE_DIRS,...} for the entire
> package including the components; this hasn't been obvious to me.

Correct.

> Nevertheless, what do you understand by "accumulative" in this regard,
> i.e. what accumulates in which variables if FIND_PACKAGE() is invoked
> multiple times and each component has an independent set of variables
> separate from the package's base stuff?

Ok, "accumulative" maybe wasn't the best choice of words. What I mean it 
that subsequent find_package calls don't overwrite previous values.
I.e. the components accumulate, not the contents of some variable.

In other words, I would expect these two statements:

find_package(XXX REQUIRED YYY)
find_package(XXX COMPONENTS ZZZ)

to give me a working environment in which I can safely use XXX, YYY and,
if XXX_ZZZ_FOUND is true, ZZZ.

>> If it was otherwise, then target AAA would be overlinked, and BBB would
>> probably fail to link. Even without the ADD_SUBDIRECTORY the situation
>> would be far from ideal: in order to avoid overlinking of BBB I have to
>> make an additional find_package invocation. In a worst-case scenario one
>> would need multiple find_package invocations for the same packages before
>> each target!
> 
> At first, BBB could not fail to link due to the FIND_PACKAGE() in the
> subdir/CMakeLists.txt because the XXX_LIBRARIES variable in the parent
> directory is not modified unless the find module uses the PARENT_SCOPE
> option of SET(), but there's no reason to do so as it is pretty unwise.

You are right. I didn't think about the scope.

> In contrast, the potential overlinking of AAA is a point, indeed, as it
> is generally with multiple targets, each depending on an individual set
> of components, within the same CMakeLists.txt. Here, I'd actually need
> to call FIND_PACKAGE() multiple times and immediately process/save the
> variables like XXX_LIBRARIES if they're really invocation-specifically
> populated, but I'm in doubt whether this can be reckoned as "far from
> ideal". Note that because of CMake's caching, repeated invocations of
> FIND_PACKAGE() don't tend to be expensive, so
> 
> FIND_PACKAGE(XXX COMPONENTS base)
> TARGET_LINK_LIBRARIES(AAA ${XXX_LIBRARIES})
> FIND_PACKAGE(XXX COMPONENTS base YYY)
> TARGET_LINK_LIBRARIES(BBB ${XXX_LIBRARIES})
> 
> is nothing I'd be afraid of. Moreover, in subdir/CMakeLists.txt, you
> have to call FIND_PACKAGE() anew anyway, as you've also when mixing
> optional and required components with a usage of the REQUIRED flag.
> 
>> The "accumulative scenario" of find_package without side-effects of the
>> components on the XXX_LIBRARIES is far more transparent:
>> If you link against XXX_LIBRARIES, you know exactly what you are linking
>> against, regardless of who called find_package(XXX) with whatever
>> components since you did your find_package call. You don't even have to
>> backup your XXX_LIBRARIES and other variables, because they are not
> altered.
> 
> The downside of your approach is that you have to mention component-
> specific variables at various locations: For each component denoted
> in the FIND_PACKAGE() invocation, you would usually have to mention
> 
> - XXX_*_INCLUDE_DIRS in INCLUDE_DIRECTORIES()
> - XXX_*_LIBRARIES in TARGET_LINK_LIBRARIES()
> - XXX_*_DEFINITIONS in ADD_DEFINITIONS()

Which is less work than saving all three values for each target that
I create.

> in addition to the package's base stuff, i.e. roughly speaking, with n
> components you need to reference 3+3n variables. 

These 3+3n variables have to be defined, anyways (the find module needs 
them internally, if it wants to compose XXX_INCLUDE_DIRS etc.).
Still, referencing the 3+3n variables are less work than having to
define _and_ reference 3N variables (with N being the number of targets
in your project). 

> IMO, this thwarts the
> idea of a multi-component package: Specifying components to adjust the
> well-known and officially recommended package-related variables like
> XXX_LIBRARIES. 

These variables are well-known and officially recommended for component-
less packages only. Nobody bothered to write recommendations for
component-packages yet.

> BTW, what's the essential difference between a multi-
> component package as you outline it and multiple single-component
> packages, one for each component and another one for the base?

One find_package call instead of n calls. Easier writing and housekeeping 
of find modules (you don't have to copy essentially the same code to n 
files and keep those in sync, you don't clutter your modules directory
with countless files).

BTW you can have exactly the same behaviour with multiple find modules:
when you call the base package, it resets the XXX_INCLUDE_DIRS, 
XXX_LIBRARIES and XXX_DEFINITIONS. The component files just add their 
own stuff to these libraries. So this really isn't a fundamental
difference between multi-component packages and multi single-component 
packages.


-- 
Johannes Zarl
Virtual Reality Services

Johannes Kepler University
Informationsmanagement

Altenbergerstrasze 69
4040 Linz, Austria
Phone: +43 732 2468-8321
johannes.zarl at jku.at
http://vrc.zid.jku.at










>>> IF YYY hasn't been requested explicitly, but the find module despite
>>> looks for it, XXX_YYY_FIND_{REQUIRED,QUIETLY} should be set to FALSE
>>> and TRUE, resp., so FPHSA handles the REQUIRED/QUIET flags for each
>>> component as the user probably expects, and XXX_YYY_FOUND is set up
>>> properly at the same time.
>>
>> I'm not completely sure if I understood that all correctly. As far as
>> I can see, the REQUIRED/QUIET flags are set for all components or
>> nothing. The interface of find_package simply doesn't allow a more fine
>> grained setting of those flags.
> 
> That's right, but the REQUIRED/QUIET flags cannot be applied to every
> component in the same manner in all situations. Suppose FindXXX.cmake
> looks for each of XXX's known components, say YYY and ZZZ, regardless
> if it is explicitly requested in the FIND_PACKAGE() call or not; e.g.
> FindQt4.cmake behaves this way. Now, with
> 
> FIND_PACKAGE(XXX REQUIRED YYY)
> 
> the user would not expect the find module to bail out if ZZZ is not
> found, but the absence of YYY should terminate the configuration, so
> the REQUIRED flag applies to YYY only. A similar point may be adopted
> for the QUIET flag: If it is not specified the user probably wouldn't
> want to see any messages related to components that are not requested
> explicitly, i.e. with the FIND_PACKAGE() above, one usually does not
> want to spot "-- Found XXX_ZZZ: /usr/local/lib/libXXX_ZZZ.so ...".

Ok. So QUIET gets applied for any component which has not been 
explicitly requested, and REQUIRED is only applied to those which have 
been explicitly requested. 

>> Regarding FIND_PACKAGE_HANDLE_STANDARD_ARGS: why not simply adding
>> another command? In fact, let's discuss its interface right now, and
>> then implement it:
>>
>> set(XXX_COMPONENTS "YYY;ZZZ")
>> FIND_PACKAGE_COMPONENTS_HANDLE_STANDARD_ARGS(
>>    XXX
>>    XXX_COMPONENTS
>>    DEFAULT_MSG
>>    DEFAULT_SUFFIXES
>> )
>>
>> Assuming DEFAULT_SUFFIXES to be "LIBRARIES;INCLUDE_DIRS;DEFINITIONS",
>> this would check the following variables:
>> XXX_LIBRARIES
>> XXX_INCLUDE_DIRS
>> XXX_DEFINITIONS
>> XXX_YYY_LIBRARIES
>> XXX_YYY_INCLUDE_DIRS
>> XXX_YYY_DEFINITIONS
>> XXX_ZZZ_LIBRARIES
>> XXX_ZZZ_INCLUDE_DIRS
>> XXX_ZZZ_DEFINITIONS
>>
>> Let's say that component YYY has been found, but ZZZ is not, so the
>> XXX_YYY_* variables are set, as well as the XXX_{LIBRARIES,INCLUDE_DIRS,
>> DEFINITIONS} variables.
>>
>> So regardless of the REQUIRED flag you end up with these variable-values:
>> XXX_FOUND = TRUE
>> XXX_YYY_FOUND = TRUE
>> XXX_ZZZ_FOUND = FALSE (or XXX_ZZZ-NOTFOUND, whatever you like most)
>>
>> A fatal error is raised, if REQUIRED is true and XXX_FIND_REQUIRED_ZZZ
> is true:
>>
>> find_package(XXX REQUIRED YYY)
>>  -> OK, user can trust that XXX_FOUND and XXX_YYY_FOUND are true.
>>
>> find_package(XXX COMPONENTS ZZZ)
>>  -> OK, user has to check XXX_FOUND and XXX_ZZZ_FOUND
>> find_package(XXX REQUIRED ZZZ)
>>  -> FATAL_ERROR, cmake will abort, so XXX_FOUND=true doesn't matter.
>>
>> Is this an interface that you could use? Any criticism?
> 
> What do you do if an unusually named variable is involved to indicate a
> component's presence/absence? 

Can you give an example for this?

> While FPHSA accepts arbitrary variables,
> your FIND_PACKAGE_COMPONENTS_HANDLE_STANDARD_ARGS() is restricted to
> XXX_COMPONENTS times DEFAULT_SUFFIXES.

Actually, my FPCHSA is just as mighty as the standard FPHSA. Leave the 
XXX_COMPONENTS empty and it behaves the same way as FPHSA does (at least
if you use the same prefix for all variables, which is IMO the recommended
practice. After all, polluting the namespace is bad(tm)).

> IMO, instead of developing a completely new function, one should re-use
> established code as far as possible, and the measure necessary to apply
> FPHSA to components is just to promote the "standard args" while taking
> account of the above-mentioned point:
> 
> # Handle explicitly requested component YYY:
> # Set up (or not) XXX_YYY_LIBRARY, XXX_YYY_INCLUDE_DIR, ...
> SET(XXX_YYY_FIND_REQUIRED ${XXX_FIND_REQUIRED})
> SET(XXX_YYY_FIND_QUIETLY ${XXX_FIND_QUIETLY})
> FPHSA(XXX_YYY DEFAULT_MSG XXX_YYY_LIBRARY XXX_YYY_INCLUDE_DIR ...)
> 
> # Handle non-requested component ZZZ:
> # Set up (or not) XXX_ZZZ_LIBRARY, XXX_ZZZ_INCLUDE_DIR, ...
> SET(XXX_ZZZ_FIND_REQUIRED FALSE)
> SET(XXX_ZZZ_FIND_QUIETLY TRUE)
> FPHSA(XXX_ZZZ DEFAULT_MSG XXX_ZZZ_LIBRARY XXX_ZZZ_INCLUDE_DIR ...)

The problem is that FPHSA simply wasn't created with components in mind.
As worthwhile as code-reuse is, it's not a goal in itself. Rather than
jump through hoops in order to reuse the existing interface, IMO one 
should create a good interface for the more complex use-case at hand.

Afterwards, one can create a wrapper for FPHSA which calls the new
interface and provides backwards-compatibility. This way you can have
a lean, clean interface and code-reuse.

Back to my proposed FPCHSA: My initial goal was to provide an interface
which does as much work as possible for you, maybe at the price of
restricted variable naming. So let's come up with a better interface:

I do want to restrict the possible prefix for modules, because I really
do think it is a good practice (and a practice worth enforcing) to
require a common package prefix and one prefix for each component):

FPCHSA( XXX DEFAULT_MSG LIBRARY INCLUDE_DIR DEFINITIONS
  COMPONENT YYY DEFAULT_MSG LIBRARY INCLUDE_DIR DEFINITIONS
  COMPONENT ZZZ DEFAULT_MSG INCLUDE_DIR OTHER_VAR )

This is far noisier than the initial approach, but still much easier to
read and more compact to write that the FPHSA approach outlined above
(3 lines vs. 7 lines). Even without the namespace-enforcement it would 
be readable: 

FPCHSA( XXX DEFAULT_MSG XXX_LIBRARY XXX_INCLUDE_DIR XXX_DEFINITIONS
  COMPONENT YYY DEFAULT_MSG XXX_YYY_LIBRARY XXX_YYY_INCLUDE_DIR XXX_YYY_DEFINITIONS
  COMPONENT ZZZ DEFAULT_MSG XXX_YYY_INCLUDE_DIR XXX_YYY_OTHER_VAR )

This signature should even be downwards-compatible with FPHSA.

>>>>> - Interpretation of XXX_FOUND: Config files can't set the value of this
>>>>> variable, but find modules can, so one should think about what it means
>>>>> for XXX_FOUND if a component - requested or not - hasn't been found.
>>>>
>>>> As I have written above, I don't think that the components should alter
>>>> the values of the package-wide variables (XXX_LIBRARIES etc.). The same
>>>> applies to the XXX_FOUND variable. If you search for library XXX and
>>>> component YYY, XXX is still found even if it lacks the requested
> component.
>>>> If you want to know if XXX was found, you use XXX_FOUND, if you want
> the
>>>> same for component YYY, you use XXX_YYY_FOUND.
>>>
>>> While this interpretation of XXX_FOUND is absolutely right, IMO, the
>>> intention of XXX_LIBRARIES etc. is different as I said before: These
>>> variables should comprehend all stuff necessary to use all requested
>>> components. E.g., suppose YYY needs zlib; where are you going to put
>>> ZLIB_LIBRARY from the FIND_PACKAGE(ZLIB) invocation? XXX_YYY_LIBRARY
>>> is reserved for libXXX_YYY.so or the like only. It's XXX_LIBRARIES
>>> that takes XXX_YYY_LIBRARY along with ZLIB_LIBRARY and the other
>>> components' libraries and prerequisites, so
>>
>> Just like XXX_LIBRARIES for package XXX, you should have XXX_YYY_LIBRARIES
>> for component YYY. XXX_YYY_LIBRARIES contains both XXX_YYY_LIBARY and
>> ZLIB_LIBRARIES. A user of the package XXX should never directly use
>> XXX_YYY_LIBRARY, just like he or she should never use XXX_LIBRARY.
> 
> You could handle the components' prerequisites in this manner, but how
> do you cope with inter-component dependencies? Suppose ZZZ depends on
> YYY and the user says:
> 
> FIND_PACKAGE(XXX COMPONENTS ZZZ)
> TARGET_LINK_LIBRARIES(... ${XXX_ZZZ_LIBRARIES})
> 
> Where do you put the YYY variables? Do you add them to the ZZZ ones? 

I'd add them to the ZZZ ones, because they are needed to use ZZZ. In 
contrast, I don't add them to XXX, because XXX can be used just fine
without ZZZ.

My approach just has one rule: "If c1 needs c2, add c2 to c1."

Thus, inter-package dependencies and inter-component dependencies work
exactly the same way.

> If
> so, what do you do in the case of FIND_PACKAGE(XXX COMPONENTS YYY ZZZ)?
> Do you still add the YYY variables to ZZZ's ones, or are you willing to
> differentiate between these two situations? 

Yes, I still add them. Adding any "magic" at this point would rightly
confuse any user. These two are fully equivalent:

find_package( XXX COMPONENTS YYY ZZZ)
target_link_libraries(foo ${YYY_LIBRARIES})
target_link_libraries(bar ${ZZZ_LIBRARIES})

and:

find_package( XXX COMPONENTS YYY)
target_link_libraries(foo ${YYY_LIBRARIES})
find_package( XXX COMPONENTS ZZZ)
target_link_libraries(bar ${ZZZ_LIBRARIES})

At this point, I just realised that I was being inconsistent. After all, 
the components do depend on the whole package. So YYY_LIBRARIES should
also contain XXX_LIBRARIES.

> Is this still suitable with
> numerous inter-component dependencies? With single XXX_{LIBRARIES,...}
> variables destined for all components, there are no issues like those.

You mean if the user doesn't know that ZZZ depends on YYY, he might
add both to the link libraries, therefore adding unneeded stuff to
the linker command? I see this as less of a problem than the overlinking
issue in the other case.

Regards,
  Johannes