[Qt-interest] Too many files open error on Mac OS

[Arnold Krille]

Note that I am only guessing what serge is doing. But from my expirience with 
big files (>100GB) and systems-administration:

On Wednesday 29 September 2010 10:03:41 Ross Bencina wrote:
> Scott Aron Bloom wrote:
> > Since it sounds like you are having to read them in to show them, why not
> > close them after each rendering and reload them in when necessary...
> My impression is that Serge thinks he is getting some beneft from not
> closing and reopening the files, but it's not clear to me what that benefit
> that is. I'd be interested to learn.
> 
> What benefit are you getting by keeping all the files open?
> 
> Is the time overhead from open()/close() significant compared to read() ?

All disk accesses are slow. But open/close has no additional penalty.
If you feel like the files could get deleted while your app uses them, it makes 
sense to keep them open. Or if these files are temporary files of your app in 
the first place, that is if you create/open them and keep them open in your app 
while you delete them to hide them from nosy people/apps, it makes sense to 
keep them open at all times. Otherwise it lightens the system load if you 
close the files when you don't need them.

> Perhaps there are cache benefits? (although the file cache probably isn't
> flushed on close() and your description sounds like you don't need caching
> anyway).

One file of the size serge is speaking of would benefit from caching. Loading 
300 of them is overflowing the cache pretty fast. And even with only one file, 
the main advantage of the cache is that the file stays in cache even when its 
closed (and not deleted), so this is not a reason.

> Are you reading multiple files at once to optimise disk access for
> physically adjacent blocks?

He said each file contains the data of several zoom-levels. So he is not 
reading adjacent blocks anyway even if the files where on adjacent blocks on 
disk. And if you really want to make sure the files are on adjacent blocks on 
disk (and be smarter then your filesystem and partition stuff and device-
mapper), you have to create your own file-storage like the big database 
applications sometimes do.

In summary: There is a good reason why users can't open more then 256 files 
open at the same time. If your app needs more, it will be in trouble on many 
systems where main user != administrator.

Have fun,

Arnold
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
Url : http://lists.qt-project.org/pipermail/qt-interest-old/attachments/20100929/da4b1cd3/attachment.bin