[Interest] memory fragmentation?

[Alex Malyushytskyy]

Harri Pasanen harri at mpaja.com wrote:

>>I wonder why you say Linux is less flexible?   Couldn't you just write a
custom allocator directly using sbrk()?

>>>>Also, 32 bit linux process leaves more application space free, so you
can easily reach 2Gb process size, while I recall windows XP having
issues after 1.5Gb.    It's been a while, so I might not recall all
details...


>From the top of my head with Physical Address Extension for 32 bit
Windows you can make 3GB for the user and 1GB for the kernel.
But by default:

on 32 bit windows:
XP - 1.5Gb,
windows 2000 and Vista 32 -  2Gb

32 bit application on 64 bit windows:
- large address space aware;
  4 gb
- NOT large address space aware
  2 gb

>>I wonder why you say Linux is less flexible?   Couldn't you just write a
custom allocator directly using sbrk()

First of all I apologize if it sounded offensive.
I am not guru on Linux ( all native code I ever wrote under Linux
besides Qt were some graphics routines which basically repeated
similar GDI logic ),
but person who was writing code doing allocation under Linux said that
he could not find similar solution
which would not require reallocation of the array(s) which would be
appropriate for our case.

As far as I understand after reading help  sbrk() performs an actual
allocation ( means memory is taken - process space grow  ).
So potentially if you can control memory allocation for every part of
the code we could try to allocate the array  at the end of the memory
segment memory going to grow , and keep allocated more as needed,
keeping all the other allocations in some reserved region "ahead".
But if I understood it right you will have to control memory
allocation for all dependencies to make it work.

Under the windows it works differently. Address range is reserved (no
memory allocation occurs), application does not consume any memory and
reserved/allocated block  does not have to be and the end of allocated
address space.
So if you know maximum memory your application might need/can afford
you do not even have to control memory allocation for other objects.

For example: assume you need  1 GB for a single array, and all other
data can't grow beyond total 500 MB.
First thing you can do - reserve 1 GB the earliest possible, and let
other allocation happen normally.
Application will not allocate this memory until you commit it.
You want to load 50MB file into this memory. Commit 50 MB,
You want to load another file which uses 150MB. Commit extra 100 MB.
Address space  is continues, physically no memory re-allocation
occurs, pointers are kept valid.
You do not care how 3rd party components (like dlls) do memory allocation.

Anyway thanks for comments, I might try to play with custom allocator
and custom allocator using sbrk() as you suggested :)

Regards,
    Alex


















On Wed, Aug 22, 2012 at 1:21 PM, BRM <bm_witness at yahoo.com> wrote:
>> From: Jason H <scorp1us at yahoo.com>
>
>>C++ on .NET functions as it does now, however the compiler introduces the operator of ^ as a type modifier like * (pointer)
>>^ are handles to managed objects, as * are addresses of objects. The runtime then handles dereferencing the handles for you. jsut like your compiler uses the appropriate instructions with a pointer.
>
>
> By which you mean C++/CLI, not strictly C++. You can mix the two with a .NET environment, and native C++ doesn't support the ^ operator. For that matter you can use 'new' instead of 'gcnew' as well, and now you have a mixture of what's being managed and what is not.
>
>
> $0.02
>
> Ben
>
> _______________________________________________
> Interest mailing list
> Interest at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/interest