MantisBT - ITK
View Issue Details
0005795ITKpublic2007-09-28 08:012008-01-23 15:01
Luis Ibanez 
Mathieu Malaterre 
normalminoralways
closedfixed 
 
 
0005795: Loading Dynamic Libraries in Mac (.dylib versus .so extension)
ITK is searching for files with extension ".so" under Mac.

This commit in the kwsys dynamic loader changed the
extensions from ".dylib" to ".so"
http://www.itk.org/cgi-bin/viewcvs.cgi/Utilities/kwsys/DynamicLoader.cxx?root=Insight&r1=1.6&r2=1.7 [^]

The ITK Dynamic Loader delegates its functionality to
the dynamic loader of kwsys.
No tags attached.
Issue History
2007-09-28 08:01Luis IbanezNew Issue
2007-09-28 08:03Luis IbanezNote Added: 0009318
2007-09-28 08:03Luis IbanezNote Added: 0009319
2007-09-28 08:05Luis IbanezNote Added: 0009320
2007-09-28 08:05Luis IbanezStatusnew => acknowledged
2007-09-28 08:05Luis IbanezStatusacknowledged => assigned
2007-09-28 08:05Luis IbanezAssigned To => Mathieu Malaterre
2007-09-28 08:33Bill HoffmanNote Added: 0009321
2007-10-01 10:32Sean McBrideNote Added: 0009344
2007-10-01 10:50Bill HoffmanNote Added: 0009345
2007-10-01 12:29Sean McBrideNote Added: 0009348
2007-10-01 12:57Bill HoffmanNote Added: 0009349
2007-10-02 14:58Sean McBrideNote Added: 0009362
2007-10-02 15:13Bill HoffmanNote Added: 0009363
2007-10-03 09:16Sean McBrideNote Added: 0009367
2007-10-03 09:16Sean McBrideNote Added: 0009368
2008-01-23 15:00Luis IbanezNote Added: 0010245
2008-01-23 15:00Luis IbanezStatusassigned => closed
2008-01-23 15:00Luis IbanezNote Added: 0010246
2008-01-23 15:01Luis IbanezResolutionopen => fixed

Notes
(0009318)
Luis Ibanez   
2007-09-28 08:03   
Starting on Nov 29 2006, the ITK DynamicLoader is delegating
its functionality to the kwsys DynamicLoader:
http://www.itk.org/cgi-bin/viewcvs.cgi/Code/Common/itkDynamicLoader.cxx?root=Insight&r1=1.21&r2=1.22 [^]


So, the problem is actually in kwsys
(see the bottom of this change:) Fri Mar 10 18:34:03 2006 by Mathieu
http://www.itk.org/cgi-bin/viewcvs.cgi/Utilities/kwsys/DynamicLoader.cxx?root=Insight&r1=1.6&r2=1.7 [^]


and therefore, this problem is affecting not only ITK but also
VTK and CMake itself.
(0009319)
Luis Ibanez   
2007-09-28 08:03   
There were also some commits in the kwsys dynamic loader
related to versions of MacOS:

http://www.itk.org/cgi-bin/viewcvs.cgi/Utilities/kwsys/DynamicLoader.cxx?root=Insight&r1=1.14&r2=1.14.18.1 [^]
http://www.itk.org/cgi-bin/viewcvs.cgi/Utilities/kwsys/DynamicLoader.cxx?root=Insight&r1=1.14&r2=1.15 [^]
(0009320)
Luis Ibanez   
2007-09-28 08:05   
Steve Pieper has verified that for building some Slicer modules
we need to load files with extension ".dylib".
(0009321)
Bill Hoffman   
2007-09-28 08:33   
This is the correct thing to do. A .dylib cannot be loaded with itkDynamicLoader. In cmake if you do:
add_library(foo SHARED)
This will create foo.dylib, and this CANNOT be loaded with itkDynamicLoader,
instead you have to do this:
add_library(foo MODULE)
This will create foo.so, which can be loaded with itkDynamicLoader.
(0009344)
Sean McBride   
2007-10-01 10:32   
Well, this is really not my area of expertise, but I'm pretty sure ".so" is never used on Mac OS X. From what I know, ".so" is the filename extension for ELF dynamic libraries. Mac OS X does not use ELF.

What is the purpose of this DynamicLoader code? If it is to have loadable/unloadable code, I believe using a 'loadable bundle' (aka .bundle) is the best choice.

The following doc is probably a good starting point:
http://developer.apple.com/documentation/Cocoa/Conceptual/LoadingCode/Concepts/AboutLoadableBundles.html#//apple_ref/doc/uid/20001268-BCIDBAEJ [^]

This may also be interesting:
http://lists.apple.com/archives/unix-porting/2007/Aug/msg00027.html [^]

Whatever you do, don't use the NSIsSymbolNameDefined() family of functions. They are all deprecated.
(0009345)
Bill Hoffman   
2007-10-01 10:50   
For for Mac OS X 10.2.x and earlier the NS* stuff is used, but for newer than dlopen is used. The dlopen stuff can open both .dylib and .so, but only on newer Mac OS versions. The real problem is what should cmake add_library(MODULE) do. I am thinking it should keep doing what it has been doing. If a project wants a .dylib it should not use the MODULE but rather SHARED. That project will no longer work with older mac versions.
(0009348)
Sean McBride   
2007-10-01 12:29   
Maybe "add_library(MODULE)" should make a .bundle ?

I'm somewhat curious about your assertion that "the dlopen stuff can open both .dylib and .so". Since you say so, I'm sure it is true, but I have never seen anyone use .so on Mac OS X. I also can't find any docs about it. "man ld", "man dyld", and "man gcc" do not contain ".so" anywhere, but do mention ".dylib". Xcode has no templates to create a .so, but does for creating .dylib and .bundle.

Re: the NS* APIs... sorry, I should have been clearer; I just meant that if some new solution is coded up not to write new code using those old APIs.

My understanding is that if you want to create a shared lib for linking against, then a .framework or .dylib is recommended. If you want to create a loadable code module, then .bundle is recommended. Is that what add_library(foo SHARED) and add_library(foo MODULE) correspond to, respectively?
(0009349)
Bill Hoffman   
2007-10-01 12:57   
I was surprised that dlopen could open almost anything shared, but it did on a variety of Macs here at Kitware.

The .so convention comes from other unix platforms, and can
be found here:
 
http://finkproject.org/doc/porting/shared.php [^]

My understanding was the same as yours until I tried the above test.
It would seem that apple has had a change of heart, and has made
things more consistent with windows and other unix systems. On both
of those it is very common to be able to both load and link
to a shared library. Prior to the changes made by apple, in the
original versions of Darwin, you could not load something that you linked
to. Now you can. So, the whole concept of MODULE was added to cmake
just for the mac, and now it seems to be for nothing....
(0009362)
Sean McBride   
2007-10-02 14:58   
So you're saying add_library(foo SHARED) creates MH_DYLIB type libs, and add_library(foo MODULE) creates MH_BUNDLE type libs? And for the latter you use .so simply because that extension is as good as any? If so, that seems reasonable.

So what is this bug about again? :) Does ITK have some SHARED and MODULE libs mixed up?
(0009363)
Bill Hoffman   
2007-10-02 15:13   
Yes, that about sums it up. I think Slicer/ITK has some add_library(SHARED) stuff that should be add_library(MODULE)...

That said with current Mac's, I don't see a good reason not to always use SHARED, which would mean changing kwsys::DynamicLoader to look for .so and .dylib.
(0009367)
Sean McBride   
2007-10-03 09:16   
Eric Wing provided these comments to me in an email, he says I may share them:

"I don't do a whole lot with loadable bundles, but this is my take.

Loadable plugins (aka loadable bundles aka loadable modules) don't
have an official extension on OS X. So the OS doesn't care if
its .dylib, .so, .foobar, etc. Using .dylib is common since that is
the official extension of a dynamic library. However, if intending to
use only as a plugin and not dynamic library, .so is popular since it
helps keep many Makefile/Autoconf based build systems consistent
between ELF platforms and OS X. But again, OS X doesn't really care.

I don't know much about the NSModule stuff. I know the OSG code base
used to do it that way (still might actually). It will work for
plugins using the .so extension, but I think the extension could be
anything. However, we don't use files as both plugins and dynamic
libraries in OSG. They are always either-or. So I don't actually know
if you can use them as either-or.

I think dlopen was actually available in 10.3, but Apple used a
dlcompat package to do this. I'm not sure what the repercussions of
this are. 10.4 has a native port of dlopen.

With dlopen, I'm under the impression that you may use plugins/
libraries as either-or but I'm not sure if 10.3's dlcomat layer
supports this or not.

But certainly with 10.4+, extending support to look at .dylib files
instead of just .so files makes a lot more sense since if you are
using libraries as either-or, the files will likely be .dylib's.

I have never tried or seen frameworks used with dlopen. It's
something I've been wondering about though."
(0009368)
Sean McBride   
2007-10-03 09:16   
As for "changing kwsys::DynamicLoader to look for .so and .dylib"... with 10.5 being released this month, it might be prudent to wait and see how things work under that OS.
(0010245)
Luis Ibanez   
2008-01-23 15:00   
The issue was solved by overwriting the dynamic lib extension in

Insight/Code/Common/itkObjectFactoryBase.cxx

http://www.itk.org/mailman/private/insight-developers/2008-January/009953.html [^]
(0010246)
Luis Ibanez   
2008-01-23 15:00   
The renaming of the dynamic library extension solved the problem.

For the long run, this probably should be fixed in kwsys.