[Interest] Qt5, XCB and X11

[Till Oliver Knoll]

Am 11.07.2014 um 06:31 schrieb Ian Monroe <ian at monroe.nu>:

> On Thu, Jul 10, 2014 at 9:14 PM,  <Lachlan.Hetherton at csiro.au> wrote:
>> ... However, as you
>> might i I have numerous doubts about shipping copies of libxcb and
>> libX11 with our application, not the least of which is that the libs would
>> be almost guaranteed to be loaded into memory twice (I have no idea what
>> problems this could cause).
> 
> On Windows this would cause a "DLL hell" issue and maybe this is where
> your hesitance comes from. 

No, this would not cause a "DLL Hell" on Windows: two processes linking with their distinct copy of a DLL (such that it is loaded twice into memory) also works just fine on Windows, like on any other OS that supports "shared libraries".

Except off course if the library /itself/ does access resources – such as hardware like graphic cards – in such a manner that it expects to have exclusive access to those resources (and would hence not grok if another instance would try to do the same - and I share my concerns with the OP here when it comes to "X related libs"). But that would entirely be a library specific issue and would have nothing to do with the way the OS (dynamic library loader thereof) would try to resolve dynamic libraries.

What is known as "DLL Hell" refers to the fact that many applications (used to) rely on DLLs located in "system folders" (c:\Windows\system32 and the kind...). That by itself would not be problematic – if those applications would not have the right to install their own "system libraries" into those “system folders” (or even: were expected to do so!)! And since I can remember (Windows 3.1 and onwards) this was for a very long time “good practise” (and might have even been encouraged by Microsoft itself, doing the same deployment with their own Office etc. applications).

Now what happened off course if you had one application happily installed, linking against, say, c:\Windows\system32\everyone_needs_this.dll, and then installing another application yet again deploying a c:\Windows\system32\everyone_needs_this.dll (the fine print: but this version is slightly different than the previous one), off course overwriting the previous DLL, then /that/ led you quickly into what became infamously known as “DLL Hell” – this, plus the "sloppy way" how the OS let an application resolve DLLs (keyword: PATH environment etc.).

There is no excuse: you could always (technically) safely deploy your application in a "safe manner" by deploying all your DLLs into "well known locations" (read: into (sub-)folders of your application’s folder). And resolving “plugins” with a relative path was always a bad idea (google up "DLL hijacking" and the like – even Qt had some issues when resolving for "SSL" libraries – this was again discussed a couple of weeks ago here on qt-interest, when some guy wondered why some funny SSL DLLs, located in some PATH, where picked up by his Qt application, even though he thought that "he would not be using SSL at all" (but WebKit did)).

Off course, deploying all your DLLs into your own folder somewhat defeats the purpose of DLLs in the first place ;) But this is e.g. exactly how applications on OS X are deployed! They provide all their "frameworks" and shared libraries they need in their "app bundle", except "well known system libraries" (Cocoa, ….). And no application is allowed (when submitted to the "Mac App Store") to install any library outside its App Bundle!

And also "non-Mac App Store" applications better not overwrite existing C/C++/Obj-C runtime libraries or even Cocoa libs (even though technically they could, when installed with admin rights – and that would exactly lead to the same "DLL Hell" as on Windows (or Linux, or any other system that supports shared libraries for that matter).

Then there’s the issue with the C/C++ runtime libs: off course those are just "too common" to be installed in each application’s folder. So Microsoft provides a "C/C++ Runtime Redistributable" installer which is run as part of the application installation process (or at least is supposed to be – some apps simply refer to the Microsoft website and tell the user to fetch the "C/C++ runtime redistributable" themselves – or silently install those DLLs themselves under, yes, you guessed it, "Windows system folder").

So if you are lucky and install all your applications "in the right order" (by published year would be a good guess ;)) then the runtime DLLs were always overwritten with a newer version – which hopefully would be fully backwards compatible (*cough* *cough*). But again, practise usually led to the "DLL Hell", breaking existing applications.

Microsoft overcame this issue with "Manifests" and "Private Assemblies" and what not, so you could install "different versions of the same C/C++ runtime" in parallel (c:\Windows\WinSxS\...) [1]. That worked again pretty well when you went down the official "C/C++ Runtime Redistributable" installer approach (read: if you used Microsoft’s own MSI installer) – in my past experience that however more often than not led to what I called the "Manifest Fiasko". We had our own installation framework, for given reasons, and trying to deploy the C/C++ runtime libs together with your application worked in most cases – except where it didn’t: I remember one "Japanese Windows XP" system where you got a funny error shortly after double-clicking the app – and I am not referring to some Japanese characters that I could not read ;).


In short: Windows let you mostly avoid the "DLL Hell" since a long time – it just made it particularly easy to fall into. Also by the fact that Microsoft let (forced) many application developers to distribute essential "system/runtime" libraries together with their applications (on the other hand, I have never ever seen anyone seriously considering deploying an ObjC or C++ runtime on OS X ;)).

But other systems such as Linux also share some affinity to the "DLL Hell": agreed, most developers there have learned from the very beginning that it is probably not a good idea to install their own libraries under /usr/lib – but dare you try to deploy your app on different Linux distributions! Off course the Unix/Linux folks also tried to overcome the "DLL Hell" by encoding "version info" into their libs, such as libfoo.0.4.2.so – but most apps simply don’t care about linking with –lfoo.0.4 (or whatever the syntax would be), so they’d also happily link with a libfoo.0.5.0.so, which in the best case might be binary compatible, but otherwise behave differently and *bang*! There you go: "Shared Objects Hell"!

I think the problem on Linux is less of an issue simply because you have no other choice than "building and testing on the exact same distro where you also intend to deploy" – or you provide the "do it yourself"-sources in the first place. Or let the distro owner compile and distribute the binaries for you. Or go a long mile, restrict yourself to the lowest common denominator of all the distros that you plan to support, test on a massive array of hardware/distros/versions and try to deploy your binary there (such as Qt does).

Okay, that last paragraph was a bit provocative, but you get the point ;)

But for sure "DLL Hell" does not refer to "two processes linking with their own distinct instance of a given library" – because that works just perfect on Windows as well.

Cheers, Oliver

[1] This is a good starter:http://en.wikipedia.org/wiki/Side-by-side_assembly - I just noticed that Microsoft apparently stopped with that "Side by Side" (SxS) deployment with their C/C++ runtime: from Wikipedia: "Microsoft Visual C++ 2005 and 2008 employ SxS with all C runtime libraries. However, runtime libraries in Visual C++ 2010 no longer use this technology; instead, they include the version number of a DLL in its file name, which means that different versions of one DLL will technically be completely different DLLs now" - seems like the "Manifest Fiasko" also overhelmed the folks in Redmond in the meantime ;)

[2] For completeness sake:http://en.wikipedia.org/wiki/DLL_hell
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/interest/attachments/20140711/b8d9e266/attachment.html>