View Issue Details [ Jump to Notes ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005795ITKpublic2007-09-28 08:012008-01-23 15:01
ReporterLuis Ibanez 
Assigned ToMathieu Malaterre 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version 
Summary0005795: Loading Dynamic Libraries in Mac (.dylib versus .so extension)
DescriptionITK is searching for files with extension ".so" under Mac.
Additional Information
This commit in the kwsys dynamic loader changed the
extensions from ".dylib" to ".so"
http://www.itk.org/cgi-bin/viewcvs.cgi/Utilities/kwsys/DynamicLoader.cxx?root=Insight&r1=1.6&r2=1.7 [^]

The ITK Dynamic Loader delegates its functionality to
the dynamic loader of kwsys.
TagsNo tags attached.
Resolution Date
Sprint
Sprint Status
Attached Files

 Relationships

  Notes
(0009318)
Luis Ibanez (manager)
2007-09-28 08:03

 Starting on Nov 29 2006, the ITK DynamicLoader is delegating
its functionality to the kwsys DynamicLoader:
http://www.itk.org/cgi-bin/viewcvs.cgi/Code/Common/itkDynamicLoader.cxx?root=Insight&r1=1.21&r2=1.22 [^]


So, the problem is actually in kwsys
(see the bottom of this change:) Fri Mar 10 18:34:03 2006 by Mathieu
http://www.itk.org/cgi-bin/viewcvs.cgi/Utilities/kwsys/DynamicLoader.cxx?root=Insight&r1=1.6&r2=1.7 [^]


and therefore, this problem is affecting not only ITK but also
VTK and CMake itself.
(0009319)
Luis Ibanez (manager)
2007-09-28 08:03

 There were also some commits in the kwsys dynamic loader
related to versions of MacOS:

http://www.itk.org/cgi-bin/viewcvs.cgi/Utilities/kwsys/DynamicLoader.cxx?root=Insight&r1=1.14&r2=1.14.18.1 [^]
http://www.itk.org/cgi-bin/viewcvs.cgi/Utilities/kwsys/DynamicLoader.cxx?root=Insight&r1=1.14&r2=1.15 [^]
(0009320)
Luis Ibanez (manager)
2007-09-28 08:05

 Steve Pieper has verified that for building some Slicer modules
we need to load files with extension ".dylib".
(0009321)
Bill Hoffman (developer)
2007-09-28 08:33

 This is the correct thing to do. A .dylib cannot be loaded with itkDynamicLoader. In cmake if you do:
add_library(foo SHARED)
This will create foo.dylib, and this CANNOT be loaded with itkDynamicLoader,
instead you have to do this:
add_library(foo MODULE)
This will create foo.so, which can be loaded with itkDynamicLoader.
(0009344)
Sean McBride (developer)
2007-10-01 10:32

 Well, this is really not my area of expertise, but I'm pretty sure ".so" is never used on Mac OS X. From what I know, ".so" is the filename extension for ELF dynamic libraries. Mac OS X does not use ELF.

What is the purpose of this DynamicLoader code? If it is to have loadable/unloadable code, I believe using a 'loadable bundle' (aka .bundle) is the best choice.

The following doc is probably a good starting point:
http://developer.apple.com/documentation/Cocoa/Conceptual/LoadingCode/Concepts/AboutLoadableBundles.html#//apple_ref/doc/uid/20001268-BCIDBAEJ [^]

This may also be interesting:
http://lists.apple.com/archives/unix-porting/2007/Aug/msg00027.html [^]

Whatever you do, don't use the NSIsSymbolNameDefined() family of functions. They are all deprecated.
(0009345)
Bill Hoffman (developer)
2007-10-01 10:50

 For for Mac OS X 10.2.x and earlier the NS* stuff is used, but for newer than dlopen is used. The dlopen stuff can open both .dylib and .so, but only on newer Mac OS versions. The real problem is what should cmake add_library(MODULE) do. I am thinking it should keep doing what it has been doing. If a project wants a .dylib it should not use the MODULE but rather SHARED. That project will no longer work with older mac versions.
(0009348)
Sean McBride (developer)
2007-10-01 12:29

 Maybe "add_library(MODULE)" should make a .bundle ?

I'm somewhat curious about your assertion that "the dlopen stuff can open both .dylib and .so". Since you say so, I'm sure it is true, but I have never seen anyone use .so on Mac OS X. I also can't find any docs about it. "man ld", "man dyld", and "man gcc" do not contain ".so" anywhere, but do mention ".dylib". Xcode has no templates to create a .so, but does for creating .dylib and .bundle.

Re: the NS* APIs... sorry, I should have been clearer; I just meant that if some new solution is coded up not to write new code using those old APIs.

My understanding is that if you want to create a shared lib for linking against, then a .framework or .dylib is recommended. If you want to create a loadable code module, then .bundle is recommended. Is that what add_library(foo SHARED) and add_library(foo MODULE) correspond to, respectively?
(0009349)
Bill Hoffman (developer)
2007-10-01 12:57

 I was surprised that dlopen could open almost anything shared, but it did on a variety of Macs here at Kitware.

The .so convention comes from other unix platforms, and can
be found here:
 
http://finkproject.org/doc/porting/shared.php [^]

My understanding was the same as yours until I tried the above test.
It would seem that apple has had a change of heart, and has made
things more consistent with windows and other unix systems. On both
of those it is very common to be able to both load and link
to a shared library. Prior to the changes made by apple, in the
original versions of Darwin, you could not load something that you linked
to. Now you can. So, the whole concept of MODULE was added to cmake
just for the mac, and now it seems to be for nothing....
(0009362)
Sean McBride (developer)
2007-10-02 14:58

 So you're saying add_library(foo SHARED) creates MH_DYLIB type libs, and add_library(foo MODULE) creates MH_BUNDLE type libs? And for the latter you use .so simply because that extension is as good as any? If so, that seems reasonable.

So what is this bug about again? :) Does ITK have some SHARED and MODULE libs mixed up?
(0009363)
Bill Hoffman (developer)
2007-10-02 15:13

 Yes, that about sums it up. I think Slicer/ITK has some add_library(SHARED) stuff that should be add_library(MODULE)...

That said with current Mac's, I don't see a good reason not to always use SHARED, which would mean changing kwsys::DynamicLoader to look for .so and .dylib.
(0009367)
Sean McBride (developer)
2007-10-03 09:16

 Eric Wing provided these comments to me in an email, he says I may share them:

"I don't do a whole lot with loadable bundles, but this is my take.

Loadable plugins (aka loadable bundles aka loadable modules) don't
have an official extension on OS X. So the OS doesn't care if
its .dylib, .so, .foobar, etc. Using .dylib is common since that is
the official extension of a dynamic library. However, if intending to
use only as a plugin and not dynamic library, .so is popular since it
helps keep many Makefile/Autoconf based build systems consistent
between ELF platforms and OS X. But again, OS X doesn't really care.

I don't know much about the NSModule stuff. I know the OSG code base
used to do it that way (still might actually). It will work for
plugins using the .so extension, but I think the extension could be
anything. However, we don't use files as both plugins and dynamic
libraries in OSG. They are always either-or. So I don't actually know
if you can use them as either-or.

I think dlopen was actually available in 10.3, but Apple used a
dlcompat package to do this. I'm not sure what the repercussions of
this are. 10.4 has a native port of dlopen.

With dlopen, I'm under the impression that you may use plugins/
libraries as either-or but I'm not sure if 10.3's dlcomat layer
supports this or not.

But certainly with 10.4+, extending support to look at .dylib files
instead of just .so files makes a lot more sense since if you are
using libraries as either-or, the files will likely be .dylib's.

I have never tried or seen frameworks used with dlopen. It's
something I've been wondering about though."
(0009368)
Sean McBride (developer)
2007-10-03 09:16

 As for "changing kwsys::DynamicLoader to look for .so and .dylib"... with 10.5 being released this month, it might be prudent to wait and see how things work under that OS.
(0010245)
Luis Ibanez (manager)
2008-01-23 15:00

 The issue was solved by overwriting the dynamic lib extension in

Insight/Code/Common/itkObjectFactoryBase.cxx

http://www.itk.org/mailman/private/insight-developers/2008-January/009953.html [^]
(0010246)
Luis Ibanez (manager)
2008-01-23 15:00

 The renaming of the dynamic library extension solved the problem.

For the long run, this probably should be fixed in kwsys.

 Issue History
Date Modified Username Field Change
2007-09-28 08:01 Luis Ibanez New Issue
2007-09-28 08:03 Luis Ibanez Note Added: 0009318
2007-09-28 08:03 Luis Ibanez Note Added: 0009319
2007-09-28 08:05 Luis Ibanez Note Added: 0009320
2007-09-28 08:05 Luis Ibanez Status new => acknowledged
2007-09-28 08:05 Luis Ibanez Status acknowledged => assigned
2007-09-28 08:05 Luis Ibanez Assigned To => Mathieu Malaterre
2007-09-28 08:33 Bill Hoffman Note Added: 0009321
2007-10-01 10:32 Sean McBride Note Added: 0009344
2007-10-01 10:50 Bill Hoffman Note Added: 0009345
2007-10-01 12:29 Sean McBride Note Added: 0009348
2007-10-01 12:57 Bill Hoffman Note Added: 0009349
2007-10-02 14:58 Sean McBride Note Added: 0009362
2007-10-02 15:13 Bill Hoffman Note Added: 0009363
2007-10-03 09:16 Sean McBride Note Added: 0009367
2007-10-03 09:16 Sean McBride Note Added: 0009368
2008-01-23 15:00 Luis Ibanez Note Added: 0010245
2008-01-23 15:00 Luis Ibanez Status assigned => closed
2008-01-23 15:00 Luis Ibanez Note Added: 0010246
2008-01-23 15:01 Luis Ibanez Resolution open => fixed

Copyright © 2000 - 2018 MantisBT Team