[CMake] Finding Python3

[Michael Hertling]

On 07/23/2010 09:02 AM, Michael Wild wrote:
> 
> On 23. Jul, 2010, at 6:29 , Michael Hertling wrote:
> 
>> On 07/22/2010 10:36 AM, Michael Wild wrote:
>>>
>>> On 22. Jul, 2010, at 10:17 , Marcel Loose wrote:
>>> [...]
>>>>
>>>> Hi Michael and others,
>>>>
>>>> I mostly agree with what your saying. However, IMHO, you refer to a
>>>> "perfect world" situation, where all Find modules properly use VERSION
>>>> to specify a version number and do not abuse NAMES for that.
>>>>
>>>> I know that the current discussion focuses on FindPython; hence the
>>>> subject ;-). However, in the "real world" quite a number of other Find
>>>> scripts are shipped as part of the CMake distribution that don't follow
>>>> this "perfect" scheme either.
>>>>
>>>> So the real question should be, I guess: Should CMake be fixed by
>>>> swapping the paths and names loops in the FindXXX() functions (issue
>>>> 10718)? Or should all abusing Find scripts be fixed?
>>>>
>>>> Best regards,
>>>> Marcel Loose.
>>>
>>> My question is more fundamental:
>>>
>>> How do I find the most recent version? Because that is why NAMES is being "abused" in the first place.
>>
>> That's a very good point. The simplest approach is probably to rely on
>> the assumption that the unversioned executable is the most recent one,
>> but this won't be true in any case, of course. If one doesn't want to
>> hardcode the possible versions in the correct order after the NAMES
>> option there has to be a way to teach the find module how to respect
>> such auxiliary conditions like "take the highest acceptable version".
>>
>> Earlier in this thread, I presented a function GET_PYTHON_VERSION()
>> that looks for Python libraries without querying the interpreter and
>> extracts the version from the first one found; this function can be
>> modified to return the highest version instead, e.g. as follows:
>>
>> FUNCTION(GET_PYTHON_VERSION RESULT)
>>    SET(v)
>>    FOREACH(i IN LISTS PYTHON_ROOT ARGN)
>>        FILE(GLOB j "${i}/lib/python*")
>>        FOREACH(k IN LISTS j)
>>            IF(k MATCHES "python[0-9].*\$")
>>                STRING(REGEX REPLACE
>>                    "^.*python([0-9].*)\$" "\\1" l "${k}"
>>                )
>>                IF("${l}" VERSION_GREATER "${v}")
>>                    SET(v "${l}")
>>                ENDIF()
>>            ENDIF()
>>        ENDFOREACH()
>>    ENDFOREACH()
>>    SET("${RESULT}" "${v}" PARENT_SCOPE)
>> ENDFUNCTION()
>>
>> Of course, the crucial moment is to provide the directories to search.
>> Here, the variable PYTHON_ROOT assists, and the directories examined by
>> FIND_PROGRAM() are known, so it should be possible to visit each Python
>> installation and figure out the version of the most recent amongst them.
>> Further refinements may be performed with the globbing expression, e.g.
>> FILE(GLOB j "${i}/lib/python2.*") to search for Python 2 installations
>> only, which could in turn be fed by the Python_FIND_VERSION variables.
>> Finally, the result of GET_PYTHON_VERSION() can be used in subsequent
>> calls to FIND_PROGRAM() et al. to locate the Python installation with
>> the most recent version.
>>
>> Regards,
>>
>> Michael
> 
> <gollum>Hateful, ugly function! Go away! It blinds us!</gollum>

I knew this would meet with your approval! ;)

> Essentially, you are replacing find_library, which IMHO is not desirable. [...]

What I would have to mimic to be as functional as the find commands is
their passing through the paths, but regrettably, the latter isn't at
one's proposal, so I've to do it by myself. Granted, that's annoying.

> [...] Particularly, because you are not searching all the paths find_library would (CMAKE_PREFIX_PATH, CMAKE_LIBRARY_PATH, CMAKE_FRAMEWORK_PATH, CMAKE_SYSTEM_PREFIX_PATH, CMAKE_SYSTEM_LIBRARY_PATH, CMAKE_SYSTEM_FRAMEWORK_PATH, CMAKE_FIND_ROOT_PATH). [...]

This is why I said "the directories examined by FIND_PROGRAM() are
known", so it's possible to supply them to a function like the one
above, perhaps by implementing a GET_{PROGRAM,LIBRARY,FILE}_PATH()
function which returns just the path the appropriate find command
examines, too, and IIRC, such a facility has already been asked
for in other contexts several times.

> [...] So the above function would need to be much more elaborate.

It's meant to outline the approach but not to be production-ready.

> I'm just listing a few ideas, no idea whether they are any good:
> 
> * How about globbing/regex support in find_XXX?

As I've said twice in this thread before, wildcard capabilities of the
find commands would probably ease the situation significantly. To check
the feasibility, I've added CMake's file globbing to FIND_PROGRAM() - a
patch of just 120 lines - and it seems to work fine, so implementing it
for FIND_{LIBRARY,FILE,PATH}() also shouldn't be invulnerable to attack.

Nevertheless, what to return if there're multiple hits? Currently, I
sort the candidates in decreasing order and return the first one that
exists, i.e. supposedly the highest version, but perhaps, this is not
always the right choice.

Regular expressions would possibly even suit better, but they're more
difficult to implement as their meta characters interfere with those
common in filenames; this risk is quite low with globbing. Probably,
there would be a need of an option, say, PATTERN analogous to NAMES
whereas globbing could perhaps be acceptable without such an effort.

> * How about an ALL option for find_XXX telling them to find all matching candidates? Is this desirable, or will this slow things down to a crawl?

Also difficult to implement, IMO, as all of the concerned methods in
Source/kwsys/SystemTools.cxx return a single string but no container.

> * Support for a version-regex argument à la REGEX REPLACE which allows find_XXX to sort the results and pick the newest? Peformance?

Possibly easier to implement, e.g. a further parameter of the methods.

> * An option to pass a callback function which takes two paths and returns -1, 0, 1 (again, allowing find_XXX to sort the results)? Performance???

Callbacks in CMake, that's new and interesting and worth a thought.

Regards,

Michael