[Qt-interest] Replace Qt Allocator

[Eric Clark]

> -----Original Message-----
> From: Dan White [mailto:ygor at comcast.net]
> Sent: Wednesday, September 16, 2009 9:47 AM
> To: Qt Interest (qt-interest at trolltech.com)
> Cc: Eric Clark
> Subject: Re: [Qt-interest] Replace Qt Allocator
> 
> Short Answer: Don't use nedmalloc with C++
> 
> According to what appears to be the nedMalloc homepage
> <http://www.nedprod.com/programs/portable/nedmalloc/>, it is a variant
> on the old, classic C function "malloc"

This is not an option. Nedmalloc is extremely fast and extremely optimized for threaded applications. Our application is on the order of about 150% faster when nedmalloc is used. You should read more about nedmalloc before saying that it is bad because it is not bad. What is bad is the fact that Qt is violating the ISO standard for templated classes. Of course nedmalloc is a variant of malloc, all allocators are a variant of malloc since it was the one and only allocator in the beginning. 

> 
> This is dangerously bad
> Reference: <http://www.devx.com/tips/Tip/12491>
> 
> "Avoid Using malloc() and free() in C++
> The use of malloc() and free() functions in a C++ file is not
> recommended and is even dangerous:
> 1. malloc() requires the exact number of bytes as an argument whereas
> new calculates the size of the allocated object automatically. By using
> new, silly mistakes such as the following are avoided:
> 
> 
> long * p = malloc(sizeof(short)); //p originally pointed to a short;
>  				//changed later (but  malloc's arg was not)
> 2. malloc() does not handle allocation failures, so you have to test
> the return value of malloc() on each and every call. This tedious and
> dangerous function imposes performance penalty and bloats your .exe
> files. On the other hand, new throws an exception of type
> std::bad_alloc when it fails, so your code may contain only one
> catch(std::bad_alloc) clause to handle such exceptions.
> 
> 3. As opposed to new, malloc() does not invoke the object's
> constructor. It only allocates uninitialized memory. The use of objects
> allocated this way is undefined and should never occur. Similarly,
> free() does not invoke its object's destructor."
> 
> ... which pretty much describes your problems.

This does not explain my problem at all. We never invoke malloc(), free(), or the ned versions of these directly. Everything is done by overriding the new and delete operators. We always use new and we always use delete. This is a problem in Qt, not in our application or in the implementation of nedmalloc. Qt gives the functions qFree(), qMalloc() and qRealloc() to allow the overriding of the allocators used within Qt, but reimplementing these functions is simply not enough.

As I said before, nedmalloc is extremely fast. Our application needs to be as fast as possible because it is a real time application. Simply saying not to use nedmalloc is not a solution. We have been using Qt and nedmalloc since Qt 3 and prior to Qt 4 we never had an issue. The container classes were modified dramatically in Qt 4 and due to the changes, this bug was introduced. If Qt is going to provide a means for reimplementing the allocator, there should be a full implementation, not just a partial one. Why have the qFree, qMalloc, and qRealloc functions there to allow the replacement of the system allocator if those functions are not called all the time, like they should be?

> 
> “Sometimes I think the surest sign that intelligent life exists
> elsewhere in the universe is that none of it has tried to contact us.”
> Bill Waterson (Calvin & Hobbes)
> 
> ----- Original Message -----
> From: Eric Clark <eclark at ara.com>
> To: Qt Interest (qt-interest at trolltech.com) <qt-interest at trolltech.com>
> Sent: Wed, 16 Sep 2009 14:31:42 +0000 (UTC)
> Subject: [Qt-interest] Replace Qt Allocator
> 
> Hello All,
> 
> Our program uses a system allocator called nedmalloc. We override the
> new and delete operators to use this system allocator. However, after
> porting to Qt 4 we have ran into some issues with the QList class and
> it's destructor. When the list is destructed, it attempts to free the
> memory created by the list, but nedfree() crashes because the memory
> was instantiated using the system allocator, malloc(). Here is the
> problem:
> 
> 
> The problem is indeed a violation of the ODR requirement in the ISO C++
> spec. For example in this line:
> 
> 
> 
>    QModelIndexList indexes = list->selectionModel()->selectedRows();
> 
> 
> 
> QModelIndexList exists in the local binary due to being a templated
> type based on QList, therefore its destructor is called from the local
> binary where ::operator delete has been defined to invoke nedmalloc.
> Unfortunately the initial constructor of the list (selectedRows())
> exists in one of the Qt DLL binaries where ::operator new is defined to
> invoke the system allocator. Therefore the list is being constructed
> with the system allocator and destructed with nedmalloc - this
> obviously enough provokes a segfault as ODR is violated.
> 
> 
> 
> I was told by some guys on this interest board that I should build
> nedmalloc into Qt. However, after doing so, I still get the same error.
> What I did to build nedmalloc into Qt, was I modified the body of
> qFree(), qMalloc(), and qRealloc() to use nedfree(), nedmalloc(), and
> nedRealloc() respectively. This apparently was not enough to fix the
> problem. Could anyone tell me what else needs to be done? Do I need to
> override the new and delete operators in Qt as well? Are there any
> direct calls to malloc() in the code that I am unaware of?
> 
> 
> 
> Any help with this would be greatly appreciated!
> 
> 
> 
> Eric
>