[CMake] _stl_next_prime can return garbage leading to std::length_error for vector: debug build that do not inline

[Mark Millard]

This is a suggested source code change (add a static) for an observed crash for sometypes of debug builds of cmake's ctest.

Debug builds of ctest that do not inline _stl_next_prime (which hastable.hxx does not declare as static) can have crash for Position Independent Code (PIC) using Table Of Contents (TOC) register logic for finding addresses: only one _stl_next_prime routine ends up in /usr/local/bin/test having the only two calls to get_stl_prime_list in /usr/local/bin/ctest but there being 7 or so different get_stl_prime_list routines (each with its own _stl_prime_list), each with a different Table Of Contents (TOC) register offset for a different TOC register value. (%r2 in powerpc64 FreeBSD 10.1-STABLE's ABI.)

The net result being that all but one get_stl_prime_list are never called despite the various TOC register values and only that routine's one TOC register offset is used for all the TOC register values. This results in garbage instead of prime numbers being returned form _stl_next_prime.

That in turn can lead to, for example, trying to reserve huge vectors and a std::length_error exception that stops ctest from finishing its initialization:

terminate called after throwing an instance of 'std::length_error'
what():  vector::reserve
Abort trap (core dumped)

This has been observed on powerpc64 FreeBSD 10.1-STABLE built with WITH_DEBUG (in FreeBSD terms). This environemtn uses TOC's and PIC by default. Of course the garage values could have other consequences that are not as obviously-wrong.

Declaring _stl_next_prime in hastable.hxx to also be static like get_stl_prime_list would pair the two routines in each compilation-unit --independent of if inline is being ignored or not.

===
Mark Millard
markmi at dsl-only.net