[CMake] providing library information, what's the cmake way

[Michael Hertling]

On 11/30/2010 01:32 PM, Johannes Zarl wrote:

>>>> - Do multiple consecutive FIND_PACKAGE(XXX ...) invocations act in an
>>>> accumulative manner on the invocation-specific variables?
>>>
>>> Generally speaking, I would say: yes. At least this is the way of the
>>> least surprise for the user, as in sufficiently complex projects a
>>> package may be included at different places with different arguments.
>>
>> For the same reason, I'd tend to the opposite; look at the following:
>>
>> FIND_PACKAGE(XXX COMPONENTS YYY)
>> ...
>> ADD_SUBDIRECTORY(subdir)
>>
>> In subdir/CMakeLists.txt:
>>
>> FIND_PACKAGE(XXX COMPONENTS ZZZ)
>> ...
>> TARGET_LINK_LIBRARIES(... ${XXX_LIBRARIES})
>>
>> Here, the target is also linked against XXX_YYY_LIBRARY as the latter is
>> inherited via XXX_LIBRARIES from the parent directory, but if the target
>> only needs XXX_ZZZ_LIBRARY it will possibly be overlinked. Although this
>> can be avoided by resetting XXX_LIBRARIES before each FIND_PACKAGE()
>> invocation, I don't think that's the way one would like to go.
>>
>> IMO, the invocation-specific results of any FIND_PACKAGE() call should
>> depend solely on the parameters passed in and the well-known variables
>> like CMAKE_PREFIX_PATH. The downside is that one must possibly process
>> or save the results before they could be overwritten, e.g.:
>>
>> FIND_PACKAGE(XXX REQUIRED YYY)
>> SET(LIBRARIES ${XXX_LIBRARIES})
>> FIND_PACKAGE(XXX COMPONENTS ZZZ)
>> LIST(APPEND LIBRARIES ${XXX_LIBRARIES})
>> ...
>> TARGET_LINK_LIBRARIES(... ${LIBRARIES})
>>
>> Since such related calls to FIND_PACKAGE(XXX ...) usually occur nearby
>> each other within the same CMakeLists.txt, this little penalty should
>> be acceptable whereas accumulated XXX_LIBRARIES and the like may have
>> far reaching and surprising effects, especially in complex projects.
>
> Funny enough, my example is almost the same:
>
> FIND_PACKAGE(XXX COMPONENTS YYY)
> ...
> ADD_SUBDIRECTORY(subdir)
> ...
> TARGET_LINK_LIBRARIES(AAA ${XXX_LIBRARIES})
> TARGET_LINK_LIBRARIES(BBB ${XXX_LIBRARIES} ${XXX_YYY_LIBRARIES})
>
> In subdir/CMakeLists.txt:
>
> FIND_PACKAGE(XXX COMPONENTS ZZZ)
> ...
> TARGET_LINK_LIBRARIES(subBBB ${XXX_LIBRARIES} ${XXX_ZZZ_LIBRARIES})
>
> As I mentioned above, I would expect XXX_LIBRARIES to contain only the
> base library, and find_package calls to act accumulatively.

If I understand correctly, you'd like to have a set of variables like
XXX_*_{LIBRARIES,INCLUDE_DIRS,...} for each component instead of one
comprehensive set XXX_{LIBRARIES,INCLUDE_DIRS,...} for the entire
package including the components; this hasn't been obvious to me.

Nevertheless, what do you understand by "accumulative" in this regard,
i.e. what accumulates in which variables if FIND_PACKAGE() is invoked
multiple times and each component has an independent set of variables
separate from the package's base stuff?

> If it was otherwise, then target AAA would be overlinked, and BBB would
> probably fail to link. Even without the ADD_SUBDIRECTORY the situation
> would be far from ideal: in order to avoid overlinking of BBB I have to
> make an additional find_package invocation. In a worst-case scenario one
> would need multiple find_package invocations for the same packages before
> each target!

At first, BBB could not fail to link due to the FIND_PACKAGE() in the
subdir/CMakeLists.txt because the XXX_LIBRARIES variable in the parent
directory is not modified unless the find module uses the PARENT_SCOPE
option of SET(), but there's no reason to do so as it is pretty unwise.

In contrast, the potential overlinking of AAA is a point, indeed, as it
is generally with multiple targets, each depending on an individual set
of components, within the same CMakeLists.txt. Here, I'd actually need
to call FIND_PACKAGE() multiple times and immediately process/save the
variables like XXX_LIBRARIES if they're really invocation-specifically
populated, but I'm in doubt whether this can be reckoned as "far from
ideal". Note that because of CMake's caching, repeated invocations of
FIND_PACKAGE() don't tend to be expensive, so

FIND_PACKAGE(XXX COMPONENTS base)
TARGET_LINK_LIBRARIES(AAA ${XXX_LIBRARIES})
FIND_PACKAGE(XXX COMPONENTS base YYY)
TARGET_LINK_LIBRARIES(BBB ${XXX_LIBRARIES})

is nothing I'd be afraid of. Moreover, in subdir/CMakeLists.txt, you
have to call FIND_PACKAGE() anew anyway, as you've also when mixing
optional and required components with a usage of the REQUIRED flag.

> The "accumulative scenario" of find_package without side-effects of the
> components on the XXX_LIBRARIES is far more transparent:
> If you link against XXX_LIBRARIES, you know exactly what you are linking
> against, regardless of who called find_package(XXX) with whatever
> components since you did your find_package call. You don't even have to
> backup your XXX_LIBRARIES and other variables, because they are not
altered.

The downside of your approach is that you have to mention component-
specific variables at various locations: For each component denoted
in the FIND_PACKAGE() invocation, you would usually have to mention

- XXX_*_INCLUDE_DIRS in INCLUDE_DIRECTORIES()
- XXX_*_LIBRARIES in TARGET_LINK_LIBRARIES()
- XXX_*_DEFINITIONS in ADD_DEFINITIONS()

in addition to the package's base stuff, i.e. roughly speaking, with n
components you need to reference 3+3n variables. IMO, this thwarts the
idea of a multi-component package: Specifying components to adjust the
well-known and officially recommended package-related variables like
XXX_LIBRARIES. BTW, what's the essential difference between a multi-
component package as you outline it and multiple single-component
packages, one for each component and another one for the base?

>>>> When turning towards find modules, the situation becomes even more
>>>> complicated. From a user's perspective, find modules and config files
>>>> should behave the same, but the formers can't know which components are
>>>> available, so they must look for them; this gives rise to questions
like:
>>>>
>>>> - Meaning of the REQUIRED and QUIET options: When a find module looks
>>>> for any component the effects of REQUIRED and QUIET depend on whether
>>>> the component has been requested explicitly or not. If it hasn't, the
>>>> find module mustn't bail out if the component isn't found even though
>>>> REQUIRED was specified; that's the opposite of what is expected for an
>>>> explicitly requested component. E.g., if FIND_PACKAGE(XXX REQUIRED YYY)
>>>> also looks for ZZZ on its own behalf the find module is not expected to
>>>> throw a fatal error if ZZZ is absent whereas the absence of YYY should
>>>> result in such a behaviour right away. A similar view may be taken for
>>>> the QUIET option: Even if it wasn't specified the user wouldn't expect
>>>> any messages related to components that weren't requested explicitly.
>>>
>>> This isn't that complicated. I have written such a FindLibrary.cmake
module
>>> (although without dependencies) and the REQUIRED and QUIET options are
>>> quite easy to handle. If you split the process into 2 parts, you
lose much
>>> of the complexity: first, search for any components you can find, second
>>> check for the required ones, giving diagnostic/error messages as is
>>> appropriate.
>>
>> Using two loops would indeed allow to handle the REQUIRED/QUIET flags
>> easily, but I would like to apply FIND_PACKAGE_HANDLE_STANDARD_ARGS()
>> to components in order to process them in the same standardized and
>> unique manner as FPHSA does with single-component packages. This is
>> even more interesting as FPHSA has been enhanced recently and is now
>> quite complicated so it's not an option anymore to provide a similar
>> function for components only. Since FPHSA takes care of setting the
>> FOUND variable, issuing messages and bailing out if necessary at one
>> go, the two-loop approach doesn't suit, IMO. Instead, I'd prefer to
>> promote the REQUIRED/QUIET flags to XXX_YYY_FIND_{REQUIRED,QUIETLY}
>> - if YYY has been requested explicitly - and call FPHSA like, e.g.:
>>
>> FIND_PACKAGE_HANDLE_STANDARD_ARGS(
>>     XXX_YYY
>>     DEFAULT_MSG
>>     XXX_YYY_LIBRARY XXX_YYY_INCLUDE_DIR
>> )
>>
>> IF YYY hasn't been requested explicitly, but the find module despite
>> looks for it, XXX_YYY_FIND_{REQUIRED,QUIETLY} should be set to FALSE
>> and TRUE, resp., so FPHSA handles the REQUIRED/QUIET flags for each
>> component as the user probably expects, and XXX_YYY_FOUND is set up
>> properly at the same time.
>
> I'm not completely sure if I understood that all correctly. As far as
> I can see, the REQUIRED/QUIET flags are set for all components or
> nothing. The interface of find_package simply doesn't allow a more fine
> grained setting of those flags.

That's right, but the REQUIRED/QUIET flags cannot be applied to every
component in the same manner in all situations. Suppose FindXXX.cmake
looks for each of XXX's known components, say YYY and ZZZ, regardless
if it is explicitly requested in the FIND_PACKAGE() call or not; e.g.
FindQt4.cmake behaves this way. Now, with

FIND_PACKAGE(XXX REQUIRED YYY)

the user would not expect the find module to bail out if ZZZ is not
found, but the absence of YYY should terminate the configuration, so
the REQUIRED flag applies to YYY only. A similar point may be adopted
for the QUIET flag: If it is not specified the user probably wouldn't
want to see any messages related to components that are not requested
explicitly, i.e. with the FIND_PACKAGE() above, one usually does not
want to spot "-- Found XXX_ZZZ: /usr/local/lib/libXXX_ZZZ.so ...".

> Regarding FIND_PACKAGE_HANDLE_STANDARD_ARGS: why not simply adding
> another command? In fact, let's discuss its interface right now, and
> then implement it:
>
> set(XXX_COMPONENTS "YYY;ZZZ")
> FIND_PACKAGE_COMPONENTS_HANDLE_STANDARD_ARGS(
>    XXX
>    XXX_COMPONENTS
>    DEFAULT_MSG
>    DEFAULT_SUFFIXES
> )
>
> Assuming DEFAULT_SUFFIXES to be "LIBRARIES;INCLUDE_DIRS;DEFINITIONS",
> this would check the following variables:
> XXX_LIBRARIES
> XXX_INCLUDE_DIRS
> XXX_DEFINITIONS
> XXX_YYY_LIBRARIES
> XXX_YYY_INCLUDE_DIRS
> XXX_YYY_DEFINITIONS
> XXX_ZZZ_LIBRARIES
> XXX_ZZZ_INCLUDE_DIRS
> XXX_ZZZ_DEFINITIONS
>
> Let's say that component YYY has been found, but ZZZ is not, so the
> XXX_YYY_* variables are set, as well as the XXX_{LIBRARIES,INCLUDE_DIRS,
> DEFINITIONS} variables.
>
> So regardless of the REQUIRED flag you end up with these variable-values:
> XXX_FOUND = TRUE
> XXX_YYY_FOUND = TRUE
> XXX_ZZZ_FOUND = FALSE (or XXX_ZZZ-NOTFOUND, whatever you like most)
>
> A fatal error is raised, if REQUIRED is true and XXX_FIND_REQUIRED_ZZZ
is true:
>
> find_package(XXX REQUIRED YYY)
>  -> OK, user can trust that XXX_FOUND and XXX_YYY_FOUND are true.
>
> find_package(XXX COMPONENTS ZZZ)
>  -> OK, user has to check XXX_FOUND and XXX_ZZZ_FOUND
> find_package(XXX REQUIRED ZZZ)
>  -> FATAL_ERROR, cmake will abort, so XXX_FOUND=true doesn't matter.
>
> Is this an interface that you could use? Any criticism?

What do you do if an unusually named variable is involved to indicate a
component's presence/absence? While FPHSA accepts arbitrary variables,
your FIND_PACKAGE_COMPONENTS_HANDLE_STANDARD_ARGS() is restricted to
XXX_COMPONENTS times DEFAULT_SUFFIXES.

IMO, instead of developing a completely new function, one should re-use
established code as far as possible, and the measure necessary to apply
FPHSA to components is just to promote the "standard args" while taking
account of the above-mentioned point:

# Handle explicitly requested component YYY:
# Set up (or not) XXX_YYY_LIBRARY, XXX_YYY_INCLUDE_DIR, ...
SET(XXX_YYY_FIND_REQUIRED ${XXX_FIND_REQUIRED})
SET(XXX_YYY_FIND_QUIETLY ${XXX_FIND_QUIETLY})
FPHSA(XXX_YYY DEFAULT_MSG XXX_YYY_LIBRARY XXX_YYY_INCLUDE_DIR ...)

# Handle non-requested component ZZZ:
# Set up (or not) XXX_ZZZ_LIBRARY, XXX_ZZZ_INCLUDE_DIR, ...
SET(XXX_ZZZ_FIND_REQUIRED FALSE)
SET(XXX_ZZZ_FIND_QUIETLY TRUE)
FPHSA(XXX_ZZZ DEFAULT_MSG XXX_ZZZ_LIBRARY XXX_ZZZ_INCLUDE_DIR ...)

>>>> - Interpretation of XXX_FOUND: Config files can't set the value of this
>>>> variable, but find modules can, so one should think about what it means
>>>> for XXX_FOUND if a component - requested or not - hasn't been found.
>>>
>>> As I have written above, I don't think that the components should alter
>>> the values of the package-wide variables (XXX_LIBRARIES etc.). The same
>>> applies to the XXX_FOUND variable. If you search for library XXX and
>>> component YYY, XXX is still found even if it lacks the requested
component.
>>> If you want to know if XXX was found, you use XXX_FOUND, if you want
the
>>> same for component YYY, you use XXX_YYY_FOUND.
>>
>> While this interpretation of XXX_FOUND is absolutely right, IMO, the
>> intention of XXX_LIBRARIES etc. is different as I said before: These
>> variables should comprehend all stuff necessary to use all requested
>> components. E.g., suppose YYY needs zlib; where are you going to put
>> ZLIB_LIBRARY from the FIND_PACKAGE(ZLIB) invocation? XXX_YYY_LIBRARY
>> is reserved for libXXX_YYY.so or the like only. It's XXX_LIBRARIES
>> that takes XXX_YYY_LIBRARY along with ZLIB_LIBRARY and the other
>> components' libraries and prerequisites, so
>
> Just like XXX_LIBRARIES for package XXX, you should have XXX_YYY_LIBRARIES
> for component YYY. XXX_YYY_LIBRARIES contains both XXX_YYY_LIBARY and
> ZLIB_LIBRARIES. A user of the package XXX should never directly use
> XXX_YYY_LIBRARY, just like he or she should never use XXX_LIBRARY.

You could handle the components' prerequisites in this manner, but how
do you cope with inter-component dependencies? Suppose ZZZ depends on
YYY and the user says:

FIND_PACKAGE(XXX COMPONENTS ZZZ)
TARGET_LINK_LIBRARIES(... ${XXX_ZZZ_LIBRARIES})

Where do you put the YYY variables? Do you add them to the ZZZ ones? If
so, what do you do in the case of FIND_PACKAGE(XXX COMPONENTS YYY ZZZ)?
Do you still add the YYY variables to ZZZ's ones, or are you willing to
differentiate between these two situations? Is this still suitable with
numerous inter-component dependencies? With single XXX_{LIBRARIES,...}
variables destined for all components, there are no issues like those.

Regards,

Michael