[Development] Backporting the "stop unloading plugins" change to 5.6

[Edward Welbourne]

Liang Jian said:
>    I would not to say "still reachable" simply means the allocation is
> not leaked.

Whether you call it a leak or not is just a matter of labelling; what
matters is resource use.  Does it increase without limit ?  If it
doesn't, I'm not too fussed whether you call it a leak: it would, of
course, be nice to use less of any resource, but a bounded amount of
one-off resource use (that exit() releases) isn't a big deal (unless
it's a very big allocation).

>    Think about the following senario: A programer allocate a big chunk
> of memory from heap and hold it in a global variable, supposed that by
> design this memory chunk will not used any more at some point and is
> aimed to be released to save memory, but the programmer forget to free
> it, this memory will be considered "still reachable" on exit but the
> fact is that this behaviour do harm to the program since it will waste
> more memory when running the program. This allocation should be
> considered as leak.

It is, of course, always desirable to release memory once one can:
however, this is not so desirable as to make it worth crashing to catch
every last byte.

>    Maybe the "still reachable" memory allocations caused by not
> destructing global variables of qt plugin will not do any harm to qt
> app, but others may do harm, how can we easily distinguish the bad
> ones from the good ones? I think the best approach is to make sure we
> release all memory allocated from heap on exit.

The only value of doing deallocations on exit is to make it possible to
catch leaks; and good leak-detection tools distinguish between
still-referenced resources and resources the program has allocated and
forgotten the details it would need in order to release.  In any case,
as long as a given still-held-on-exit resource shall be released by exit
(and there's only one of it), one can record this in prior runs and
ignore it in the tool's reports for later runs.  ISTR valgrind can even
be configured to filter out expected leaks, so that the ignoring is
automatic.

While it is good hygiene to make sure one *has* kept track of the means
to release all resources allocated - and one can most robustly be sure
of this by actually releasing them - it is not really necessary to
release all resources, when exit takes care of doing that for you.  The
problem isn't that we're holding onto resources on exit; it's that we're
using more resources than we should; and this - as Thiago points out -
isn't really a big deal unless there's a way for it to cycle, adding
ever more allocations that won't be released.

>    BTW, not every memory leak detector can tell you which memory is
> "still reachable", I used bulitin memory leak detector of debug
> version of msvcrt, inspector from Intel and C++ memory leak detector
> from softwareverify.com<http://softwareverify.com>, they don't have
> the concept of "still reachable".

Then they're failing at an important part of what their tool should be
doing.  Encourage them to do better, or switch to Valgrind (which is
free, as in liberty, and not just gratis for the first thirty days).

>    Anyway, I still consider that we should introduce a method to
> disable the not unloading qt plugin feature at runtime, maybe we can
> only disable this feature in a developer build of qt and that will not
> cause any crash in a release build.

Introducing a method to dlclose() won't save you from leaks on exit;
it'll just make exit happen at some random earlier time, before the user
asked for it, when some attempt is made to (release or) exercise a
resource created by the plugin, on finding that the method to do that
(or some static data it needs to reference while doing it) has been
unloaded.  So the resource can't be released - you can crash trying, or
not try.

It *is* frustrating: plugin architectures where you can load and unload
do make it possible to upgrade plugins on the fly, which is a very cool
thing to do in a long-running application.  (I've worked with the code
to do this in a web server, for example - we had a process that aimed
for (and delivered) uptimes measured in years; getting it to handle DST
transitions in its log files was one of the fun riddles I got to solve -
so being able to use later releases of a plugin was actually quite
important.)  The design constraints that make that possible, however,
tend to limit one's freedom to do creative O-O things with the plugins
(that web server was written in C); maybe it'd be possible to design a
better plugin architecture, but we're stuck with one that can't unload
safely.  Too many things it's reasonable for a plugin author to do (in
C++) are apt to make unloading fatal - and we have lots of crash bugs to
prove it,

	Eddy.