[Development] Qt 4.x and Qt 5 frameworks should use @rpath (QTBUG-31814)

Tue Aug 12 10:56:01 CEST 2014

On Aug 11, 2014, at 11:15 PM, Jake Petroules <jake.petroules at petroules.com> wrote:

> On 2014-08-11, at 10:42 AM, Ziller Eike <Eike.Ziller at digia.com> wrote:
> 
>>>> 
>>>>> In a terminal session, simply:
>>>>> 
>>>>> $ export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:/path/to/Qt/Frameworks
>>>>> $ open /apps/MyApp.app
>>>>> 
>>>>> In Creator, just click run and this will be handled automatically. In qbs just type `qbs run -p foo` and this will be handled automatically.
>>>> 
>>>> This won't work for apps launched directly via Finder/Dock.
>>> 
>>> Like I said, that doesn't matter. It didn't work in Qt 4 and the fact that it works in Qt 5 is a mere side effect. No one will notice.
>> 
>> Qt 4 worked exactly like Qt 5 does in this regard.
> 
> I think your memory might be off.

>From the “user” perspective no.

> server:~ jakepetroules$ otool -L /Library/Frameworks/QtGui.framework/Versions/Current/QtGui
> /Library/Frameworks/QtGui.framework/Versions/Current/QtGui:
> 	/usr/local/Trolltech/Qt-4.8.6/lib/QtGui.framework/Versions/4/QtGui (compatibility version 4.8.0, current version 4.8.6)

That’s the absolute path to the default install prefix when building Qt 4.
If you do a -developer-build, the install prefix is the build directory, and the id is the corresponding absolute path to the lib.
The Qt 4 binary installer actually installs Qt libraries that have a relative path as id and references, e.g. 

/Library/Frameworks/QtGui.framework/QtGui:
	QtGui.framework/Versions/4/QtGui (compatibility version 4.8.0, current version 4.8.6)
	QtCore.framework/Versions/4/QtCore (compatibility version 4.8.0, current version 4.8.6)
	/System/Library/Frameworks/Carbon.framework/Versions/A/Carbon (compatibility version 2.0.0, current version 152.0.0)

Which works because they are installed in a default search path.
That’s the installer from http://download.qt-project.org/official_releases/qt/4.8/4.8.6/ , so I have no idea how you got the install that you see.

So, it was never necessary to set any environment variables or to run your application through an IDE with Qt 4 on Mac, it was always possible to just run your application through Finder etc.

> 	/usr/local/Trolltech/Qt-4.8.6/lib/QtCore.framework/Versions/4/QtCore (compatibility version 4.8.0, current version 4.8.6)
> 	/System/Library/Frameworks/Carbon.framework/Versions/A/Carbon (compatibility version 2.0.0, current version 152.0.0)
> 	/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.3)
> 	/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit (compatibility version 45.0.0, current version 1038.0.0)
> 	/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.9.0)
> 	/usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 625.0.0)
> 	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 123.0.0)
> 	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 227.0.0)
> 	/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 44.0.0)
> 	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 550.0.0)
> 	/System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices (compatibility version 1.0.0, current version 38.0.0)
> 	/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 751.0.0)
> server:~ jakepetroules$ stat /usr/local/Trolltech/Qt-4.8.6
> stat: /usr/local/Trolltech/Qt-4.8.6: stat: No such file or directory
> 
> Notice the install name prefix differs from the actual installation directory. It might be the case that it worked only because Qt 4 installed by default in the system-recognized /Library/Frameworks and dyld falls back to that path (among others) despite a completely different absolute soname. (Odd behaviour in any case!)

>>> Most people build Qt apps on OS X using some IDE, most likely Qt Creator. Are you seriously telling me you click build, go find the build directory, and launch the app in Finder?
>> 
>> I very often start my developed app (Qt Creator) from Finder/Spotlight/Launchbar. For example when I want to show someone something, and I currently do not have Qt Creator open, or have Qt Creator open with a different application or branch of the application.
> 
> Yeah well with the "Phase II" changes (copy frameworks to the bundle) this point becomes moot.

Yes, that’s good. I just wanted to clarify what is ridiculous and what not.

> Also, nothing stops you adding an absolute runpath search path to your own app (QMAKE_RPATHDIR += $$[QT_INSTALL_LIBS]), just that qmake must NOT do this automatically.
> 
>>> That's ridiculous. You click the run button. If you're willing to go through all the former effort you can easily export DYLD_LIBRARY_PATH at the beginning of your terminal session and type `open MyApp.app`. Note that the open command uses launch services which is the same as double clicking the bundle in Finder.
>> 
>> I’m pretty sure that
>> 
>> export DYLD_LIBRARY_PATH=......
>> open Foo.app
>> 
>> does NOT work. “open” runs the application in the launchd environment.
> 
> Yes it does. (Why do so many people think otherwise?)

I stand corrected, good to know.
Maybe it is because of the documentation ( "The open command opens a file (or a directory or URL), just as if you had double-clicked the file's icon.” ), though I also think that it did not work some time that I tried it. But maybe it was just assumptions because that is what it claims to do.

> server:~ jakepetroules$ cd /tmp
> server:tmp jakepetroules$ nano main.c
> server:tmp jakepetroules$ mkdir -p foo.app/Contents/MacOS
> server:tmp jakepetroules$ clang -install_name @rpath/bar.dylib -o bar.dylib -dynamiclib main.c
> server:tmp jakepetroules$ clang -o foo.app/Contents/MacOS/foo main.c bar.dylib
> server:tmp jakepetroules$ open foo.app
> LSOpenURLsWithRole() failed with error -10810 for the file /private/tmp/foo.app.
> server:tmp jakepetroules$ DYLD_LIBRARY_PATH=$PWD open foo.app
> server:tmp jakepetroules$ 
> 
>>> 
>>> Furthermore, if you simply copy frameworks into the bundle at build time like every other native OS X app on the planet, this becomes a non-issue and you don't even need DYLD_LIBRARY_PATH. With my QMAKE_EMBEDDED_FRAMEWORKS suggestion, too, there would be a choice between the two, and be closer to native tools behaviour at the same time.
>> 
>> Copying the Qt frameworks at build time possibly would be the best thing to do (and is probably what you do when using some “native” OS X frameworks out there).
> 
> <strike>probably</strike> Yes, that's the ultimate goal.
> 
>> If we’d start doing that, we’d want to only copy the frameworks that are actually used though. And the Qt plugins and Qt Quick imports, and only the ones that are needed/wanted.
> 
> The list of libraries/frameworks/plugins would/should be user-specified. If you want to use qmake's dependency knowledge to automatically add some defaults to that list, that's fine.
> 
> However, note that some people don't necessarily even want them copied (which means they must be able to clear that default list). If you're building a large, complex suite of applications it might be better for your use case to add an rpath like /Library/Application Support/Cool Suite/Frameworks to your client apps and install Qt and your other libraries centrally at that location (no bundled frameworks whatsoever).
> 
> A lot of the arguments that have been stated seem to forget that any serious Qt application is going to link to other libraries/frameworks as well, which also need to be copied into the bundle. Attempting to automate this too far is error prone, because you can't always determine what to copy or not simply by looking at the path or anything else. User knows best.
> 
>> I don’t see how one could argue about doing that for OS X target only though, and in the end it would mean putting all the functionality of (mac/win/android)deployqt into the build process. That would definitely be more than just adapting some rpaths/ids set in the build process and install_name call adaptions in macdeployqt.
> 
> OS X and iOS are different platforms than WinRT and Android with different requirements, so not having a *deployqt for them isn't odd at all. It's a completely different process. Keep in mind that neither windeployqt or androiddeployqt would be terribly useful for certain types of installations on those platforms, either.
> 
> Copying a framework is not the same as copying an arbitrary directory to some location. In implementation it is, but it's a specialized high level concept that needs to be treated differently.
> 
> Tools like *deployqt are completely bundle-centric and should not even exist, because this is something the build system should handle or provide a means to handle.

Not quite sure what to make out of your sentences above. You seem to agree with me that all *deployqt should then be merged into what the build system does, so I’m not sure why you point out that OS X/iOS and Android/WinRT are different platforms with different requirements ;)
The point that I tried to make was: Merging *deployqt into qmake is much more effort than just making the Qt libs relocatable on Mac (which you agreed to in a later mail as “phase I”, which I read only after writing this, so maybe we should just drop this here).

>> From the Qt + Qt Creator developer point of view it would be a bit ugly when that is done in a development build, because updating my Qt then means that I have to rebuild my Qt Creator versions to see the effects, but that could be argued to be a side-case.
> 
> Or set DYLD_LIBRARY_PATH / DYLD_FRAMEWORK_PATH to force usage of the other copy of Qt. That's why it exists.

I test Qt Creator by developing Qt Creator with it. I do not start that Qt Creator from the terminal, but just type “Qt Creator” in my favorite launch tool and select the build that I want to use.
I could probably instead set DYLD_... and use ‘open’ from the command line (with ‘open’ actually inheriting environment variables, thanks for that clarification), but on Linux we have found out that using DYLD_... or LD_... for running the IDE from which you want to develop and run other applications from, might not be the safest thing to do.

Anyhow, as I said that’s probably the non-common case, and I certainly can invent ways to opt-out of the library copying and set an absolute rpath on the Qt Creator binaries for my development builds.

>>>>> Note that Qt Creator already does the logical equivalent of what I described on Windows; the Qt libraries path is added to the PATH when the process is launched. Launching directly in Explorer doesn't work, and launching directly from Finder doesn't need to work either.
>>>> 
>>>> Currently it DOES WORK when launching via Finder, because it does hardcode absolute path, so I don't see any reason it should stop working. Also it does work like that on Linux too.
>>> 
>>> Linux is not OS X (also did you forget about $ORIGIN?). And if we're going to compare to other operating systems, currently it DOESN'T work on Windows. Never seen a complaint about it.
>> 
>> We regularly have confused Windows developers on the Qt Creator IRC/mailing list, because their apps do not start from Explorer.
>> That everyone is used to Windows development being inconvenient shouldn’t be a reason to introduce that on the other platforms too ;)
> 
> This was mostly a counterargument to people complaining against copying frameworks into the bundle. However, copying frameworks into the bundle is the correct solution, and is what must be done in the end.
> 
> It'll be nice when qbs solves all these problems with the run process being powered by an in-place installation...
> 
>> -- 
>> Eike Ziller, Senior Software Engineer - Digia, Qt
>> 
>> Digia Germany GmbH, Rudower Chaussee 13, D-12489 Berlin
>> Geschäftsführer: Mika Pälsi, Juha Varelius, Tuula Haataja
>> Sitz der Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 144331 B
> 
> -- 
> Jake Petroules - jake.petroules at petroules.com
> Chief Technology Officer - Petroules Corporation

-- 
Eike Ziller, Senior Software Engineer - Digia, Qt

Digia Germany GmbH, Rudower Chaussee 13, D-12489 Berlin
Geschäftsführer: Mika Pälsi, Juha Varelius, Tuula Haataja
Sitz der Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 144331 B