MantisBT - CMake
View Issue Details
0006157CMakeCTestpublic2007-12-13 10:442010-11-08 05:44
Sean McBride 
Bill Hoffman 
normalblockalways
closedno change required 
Mac OS XMac OS X10.5
 
 
0006157: CTest cannot test multiple CPU architectures on Mac OS X
All currently-sold Macs have 64 bit Intel CPUs. As such, they can run:
 - 32 bit Intel code (i386)
 - 64 bit Intel code (x86_64)
 - 32 bit PowerPC code (ppc) (by way of Rosetta)

They cannot run 64 bit PowePC code because Rosetta does not support that.

This is great for automated testing, because you can test 3 of your 4 architectures on one machine.

Xcode includes a unit test system called OCUnit:
http://developer.apple.com/tools/unittest.html [^]

Starting with Xcode 3, OCUnit also runs tests on all possible architectures:
http://developer.apple.com/releasenotes/DeveloperTools/RN-Xcode/index.html [^]

"Two- and Three-Way Unit Tests with Garbage Collection - Unit Tests now run as many architectures as possible on the build machine. Unit tests on Intel will attempt to run the 32-bit PowerPC unit tests in Rosetta, and tests on 64-bit machines will run both 64-bit and 32-bit architectures."

CTest seems to lack this functionality.

This makes it far far less useful on Mac OS X, and is the main reason we have (sadly) decided we are unable to use it for our own projects.
No tags attached.
Issue History
2007-12-13 10:44Sean McBrideNew Issue
2007-12-14 09:38Bill HoffmanNote Added: 0009881
2007-12-14 20:28Bill HoffmanStatusnew => acknowledged
2007-12-19 10:20Sean McBrideNote Added: 0010000
2008-02-15 10:45Bill HoffmanStatusacknowledged => assigned
2008-02-15 10:45Bill HoffmanAssigned To => Bill Hoffman
2008-02-15 13:40Sean McBrideNote Added: 0010521
2008-10-01 15:37Bill HoffmanNote Added: 0013667
2008-10-01 16:51Sean McBrideNote Added: 0013685
2010-07-05 15:34Sean McBrideNote Added: 0021245
2010-09-21 12:19David ColeNote Added: 0022298
2010-11-04 17:02David ColeNote Added: 0022862
2010-11-04 17:02David ColeStatusassigned => closed
2010-11-04 17:02David ColeResolutionopen => won't fix
2010-11-04 17:30Sean McBrideNote Added: 0022865
2010-11-04 17:30Sean McBrideStatusclosed => feedback
2010-11-04 17:30Sean McBrideResolutionwon't fix => reopened
2010-11-04 17:44David ColeNote Added: 0022866
2010-11-04 17:44David ColeNote Added: 0022867
2010-11-05 18:29Sean McBrideNote Added: 0022928
2010-11-08 05:44David ColeNote Added: 0023050
2010-11-08 05:44David ColeStatusfeedback => closed
2010-11-08 05:44David ColeResolutionreopened => no change required

Notes
(0009881)
Bill Hoffman   
2007-12-14 09:38   
How are we supposed to implement this with ctest? Ctest just runs executables. What would one have to do to be able to run an executable that is not native?
(0010000)
Sean McBride   
2007-12-19 10:20   
I confess I have not thought about how to fix this. I just wanted to bring the problem to your attention. I'll let you know if/when I have some ideas...
(0010521)
Sean McBride   
2008-02-15 13:40   
Bill, looks like this won't be too hard after all. Take a look at 'man arch'. It's exactly what is needed. It allows you to specify the desired architecture and launch the executable as that architecture.
(0013667)
Bill Hoffman   
2008-10-01 15:37   
RCH(1) BSD General Commands Manual ARCH(1)

NAME
     arch -- print architecture type

SYNOPSIS
     arch

DESCRIPTION
     The arch command displays the machine's architecture type.

SEE ALSO
     machine(1)
(0013685)
Sean McBride   
2008-10-01 16:51   
which OS is that man page from? On 10.5.5 it says:

NAME
     arch -- print architecture type or run selected architecture of a universal binary

SYNOPSIS
     arch
     arch [-h] [[-arch_name | -arch arch_name]...] prog [args ...]

DESCRIPTION
     The arch command with no arguments, displays the machine's architecture type.

     The other use of the arch command it to run a selected architecture of a universal binary. A universal binary
     contains code that can run on different architectures. By default, the operating system will select the archi-
     tecture that most closely matches the processor type. This means that an intel architecture is selected on
     intel processors and a powerpc architecture is selected on powerpc processors. A 64-bit architecuture is pre-
     ferred over a 32-bit architecture on a 64-bit processor, while only 32-bit architectures can run on a 32-bit
     processor.
(0021245)
Sean McBride   
2010-07-05 15:34   
Just to ping this bug.... Nothing much has changed as of 2.8.2. I'd still like to see this supported.

As I said, 'arch' can be used to run an executable using a specified architecture. Ex:

arch -arch i386 ls
arch -arch x86_64 ls

Whereas 'lipo' can report the available architectures, Ex:

$ lipo -info /bin/ls
Architectures in the fat file: /bin/ls are: x86_64 i386

These could together be used to examine test case executables and run them multiple times.

I guess cdash may need changes too, to understand that a test could be run twice on the same machine.
(0022298)
David Cole   
2010-09-21 12:19   
(I'm just going through all the "blocking" bugs today, and ran across this one. These are my comments...)


Do you have a concrete suggestion for how you think ctest should implement this functionality?

I don't see this as ctest's responsibility... But perhaps if you set a flag, it would make sense to have ctest automatically try to run all possible architectures via arch.

It would have to be a per-test or per-target flag, though.

If I'm running a python test, do I want to run all the archs of python? Maybe, if I have a python plugin or something.

I'm uncertain that this would add significant value to ctest. You could easily write a wrapper around add_test that does this yourself.
(0022862)
David Cole   
2010-11-04 17:02   
Closing this issue due to lack of response to my last request for a useful suggestion of how to fix this... If you come up with something, feel free to re-open it and elaborate.
(0022865)
Sean McBride   
2010-11-04 17:30   
David, sorry I didn't get back sooner...

It's difficult to give concrete suggestions on how to implement it when I'm not much of a CMake expert. :(

Indeed, it may not be ctest's responsibility, maybe it's more of a cmake or cdash thing... they're kinda all the same to me. :)

Here's the high-level goal though:
  - I have code
  - it needs to work on ppc32, ppc64, i386, and x86_64
  - I'm careful to code without endianness or size dependencies, but I want to build & test nightly
  - cmake's CMAKE_OSX_ARCHITECTURES lets me build all of them nightly. this is great!
  - how do I test what I've built? currently, although executables are built universal, only one arch is executed

Note also that I don't use CMake for my own projects, but I use it for my dependencies like VTK and ITK.

My current solution is to have several different dashboard submissions, each building just one arch and therefore running just that one arch.

Is my goal clearer now?
(0022866)
David Cole   
2010-11-04 17:44   
You ask:
how do I test what I've built? currently, although executables are built universal, only one arch is executed

Here's one way, in pseudo-cmake-code:

Replace this:
add_test("${testname}" ${testexe} ${testargs})

With this:
foreach(arch ${CMAKE_OSX_ARCHITECTURES})
  add_test("${arch}${testname}" arch -arch ${arch} ${testexe} ${testargs})
endforeach()

That way, you add 4 separate tests for a 4-way Universal binary. CTest can test all the architectures in the same build, you just need to add tests that explicitly run the ones you want.

In fact, you could even get fancy and wrap CMake's built-in add_test with your own add_test macro, and just do this automatically behind the scenes for a project that otherwise just uses the normal add_test.

If you want to learn more about CMake (enough to provide us with a well-tested patch for this feature idea as a built-in functionality rather than as a project-level add_test functionality) then that would be great. But, I just don't see us spending the time on this when there's already a perfectly reasonable way to achieve the goal.
(0022867)
David Cole   
2010-11-04 17:44   
I will close this again tomorrow unless you convince me otherwise.... :-)
(0022928)
Sean McBride   
2010-11-05 18:29   
David, hopefully one day I'll give that a try, but I don't have time currently I'm afraid.

No problem with closing this report, it was a feature suggestion that you are of course free to reject. :)
(0023050)
David Cole   
2010-11-08 05:44   
Closing as discussed in the notes...