[Development] Qt 4.x and Qt 5 frameworks should use @rpath (QTBUG-31814)

[Jake Petroules]

On 2014-08-01, at 11:13 AM, Adam Strzelecki <ono at java.pl> wrote:

> Hello,
> 
> Since 5.4 feature freeze is going to happen soon, I want to escalate long standing problem described in QTBUG-31814.
> 
> Long story short, Qt is using some nasty install prefix padding and dylib rewrite (via install_name_tool) during Qt SDK install to handle user defined Qt SDK frameworks placement. This might be OK if we were in 2005 using OSX 10.4, but since 10.5 there is @rpath that makes whole process far less complicated.

Yes, this is something that desperately needs to get done!

> Therefore there is absolutely no need for:
> 
> (1) MAKE_INSTALL_PADDING in qtsdk/packaging-tools/build_wrapper.py anyway dylibs are padded with -headerpad_max_install_names
> 
> (2) no need for MacReplaceInstallNamesOperation::relocateBinary(..) in installer-framework
> 
> (3) no need for similar relocation in macqtdeploy script, this would be especially useful in future when OSX uses ZFS's COW (no idea if HFS does already COW at block level when copying)
> 
> All we need to do is build all Qt dylibs with install_name = @rpath/pathrelative_to_lib_folder/some.dylib and make sure qmake adds following linker flags to all apps linking to Qt:

Just @rpath/QtModuleName.framework/Versions/A/QtModuleName and @rpath/qtsomethingorother.dylib

> -Wl,-rpath, at loader_path/../Frameworks
> -Wl,-rpath, at executable_path/../Frameworks
> -Wl,-rpath,/absolute/path/to/dev/Qt/Frameworks
> 
> Then macdeployqt is just about removing last rpath from binary, leaving only these relative. No need to touch ever libraries once they are built.

No - don't include the last one. macdeployqt (nor any other tool) should NOT modify the binaries at all (modifying is bad for codesigning too). This is why we have DYLD_LIBRARY_PATH (see below).

> This also makes whole SDK process installation pure copy process.
> 
> I am willing to help but this has to be changed in several projects qtsdk, installer-framework, qtbase (Qmake) in order to make it work.

I've also wanted to work on this for a while but my time is limited right now. I can definitely help with reviews though.

> Here is proposal for change in Qmake:
> 
> (1) adding QMAKE_RPATHPREFIX variable specyfying list of path prefixes that will be replaced by @rpath when linking OSX library, by default this will contain Qt SDK lib/ dir
> 
> (2) adding @rpath replacement in UnixMakefileGenerator::init2() before:
> 
> project->values("QMAKE_LFLAGS_SONAME").first() += escapeFilePath(soname);
> 
> Where prefix from QMAKE_RPATHPREFIX found in soname is replaced by "@rpath"

No, QMAKE_INSTALLNAMEPREFIX, which is set to "@rpath" (as opposed to an absolute path as is done currently).

> Once this is done qtbase Qmake project should define QMAKE_RPATHPREFIX to be Qt SDK install_prefix/lib during build process.

No replacement needs to be done. When building a Qt library, its install name is set at build time to @rpath/QtModuleName.framework/Versions/X/QtModuleName or @rpath/qtsomethingorother.dylib, there's no path to replace. When a consumer links to that library, it sets *its own* runpath search path to an array of paths appropriate for that binary (such as @loader_path/../Frameworks and @executable_path/../Frameworks). Absolute paths only start mattering at load time, not build time.

So, to summarize: the linker flags to build QtCore would be:

ld [...] -install_name @rpath/QtCore.framework/Versions/A/QtCore -rpath @loader_path/../Frameworks -rpath @executable_path/../Frameworks /Qt/clang_64/Frameworks/QtCore.framework/Versions/A/QtCore

and the linker flags to OurCoolApp which links to QtCore...

ld [...] -F/Qt/clang_64/Frameworks -framework QtCore -rpath @loader_path/../Frameworks -rpath @executable_path/../Frameworks /apps/MyApp.app/Contents/MacOS/MyApp

When we link to QtCore with -framework QtCore, the -install_name argument value we used to build QtCore gets embedded into OurCoolApp as the path we link against (see `otool -L OurCoolApp`). Then when we run OurCoolApp and try to load QtCore, the @rpath in QtCore's install name gets replaced by @executable_path/../Frameworks and @loader_path/../Frameworks, thus the runpath search path is resolved to @loader_path/../Frameworks/QtCore.framework/Versions/A/QtCore and @executable_path/../Frameworks/QtCore.framework/Versions/A/QtCore, which are then both resolved to /apps/MyApp.app/Contents/MacOS/../Frameworks/QtCore.framework/Versions/A/QtCore, thus /apps/MyApp.app/Contents/Frameworks/QtCore.framework/Versions/A/QtCore

No file exists at this path yet because we haven't copied it to our app bundle. So in development mode, Qt Creator, qbs, etc., would set an appropriate DYLD_LIBRARY_PATH to augment the existing runpath search paths (here's why we don't need your third absolute path as mentioned above). Like so:

export QT_INSTALL_LIBS=$(qmake -query QT_INSTALL_LIBS)
export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:$QT_INSTALL_LIBS
/apps/MyApp.app/Contents/MacOS/MyApp

Thus when loading QtCore.framework, resolving [@loader_path/../Frameworks, @executable_path/../Frameworks, /Qt/clang_64/Frameworks] to [/apps/MyApp.app/Contents/Frameworks/QtCore.framework/Versions/A/QtCore, /Qt/clang_64/Frameworks/QtCore.framework/Versions/A/QtCore]

QtCore is not found at the first path, but is found at the second, and we load it. When we deploy, we run macdeployqt and QtCore is found at the first search path instead. Perfect harmony. :)

What might be good, too, is a QMAKE_EMBEDDED_FRAMEWORKS or somesuch which copies the Qt frameworks into the application bundle at build time (though you'd still set DYLD_LIBRARY_PATH because not everyone would necessarily use QMAKE_EMBEDDED_FRAMEWORKS). No reason to postpone that step to macdeployqt; this is how the native Apple tools work anyways. Ossi will disagree with me here and say that anything to do with creating bundle structures is a deployment step only, though. :)

Also, I'd add @executable_path/../Libraries and @loader_path/../Libraries to the search path as well. While less commonly known/used (and found in at least some official Apple documentation), some people do like to put frameworks in $CONTENTS_FOLDER_PATH/Frameworks and dylibs in $CONTENTS_FOLDER_PATH/Libraries.

> 
> WDYT?
> 
> Regards,
> -- 
> Adam Strzelecki
> 
> _______________________________________________
> Development mailing list
> Development at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development


QLibraryInfo::location / qt.conf and such will need to be looked at as well to properly support an @rpath build of Qt. Thiago knows more about this.
-- 
Jake Petroules - jake.petroules at petroules.com
Chief Technology Officer - Petroules Corporation