[Development] Default enabling inbound flow control on sockets

Olivier Goffart olivier at woboq.com
Mon Feb 13 19:57:30 CET 2012


On Monday 13 February 2012 18:33:51 marius.storm-olsen at nokia.com wrote:
> On 02/13/2012 08:10 AM, ext shane.kearns at accenture.com wrote:
> >> Not sure. Is it a big problem? Or is it better to just continue as
> >> is, and let the applications that do have a problem set it to
> >> something reasonable to them instead?
> >> 
> >> I'd probably suggest that we instead improve the output on that
> >> worst case failure to help devs fix the problems in their
> >> programs.
> >> 
> >> $0.02
> >> 
> >> Ben
> > 
> > qWarning allocates memory, so once the worst case happens, we can't
> > give any output (unless we first handle the exceptions inside
> > Q*Socket) It would be possible to print a warning when buffering
> > exceeds some threshold we consider to be unreasonably large. We can
> > also improve the documentation (it's not bad, but doesn't mention
> > flow control explicitly)
> > 
> > https://bugreports.qt-project.org/browse/QTBUG-14464 seems to be
> > explained by missing flow control causing out of memory errors under
> > load.
> 
> Qt is not known for handling OOM cases, and we actually argue for the OS
> to handle it, and that the application (and its libraries) shouldn't
> have to think about memory not being available.
> (Too many error conditions and OOM handling all over the place to bring
> any real benefit for it anyways.)
> 
> Here you are suggesting a behavior change to handle an OOM case, where
> the behavior will most likely cause valid applications to stop working,
> since they don't handle a (new and artificial) 64k in-buffer limit?
> 
> Maybe we should rather allocate a buffer for qWarning output, allowing
> us to properly report an error condition before the application aborts?
> 
> ^shrug^ but I rather not have applications start failing at random due
> to a failure to have a new limit.

Here the problem is not really OOM, but bad putting way too much data 
structure into memory.

QByteArray is not designed to contain huge data (2GiB is really the maximum, 
only because we use int for indexes)

and stuff like readAll() will crash if that is hapenning.

The second problem seems to be that QTCPSocket buffers everything, it does not 
matters if the program does anything with it or just let the event loop spin.
If QTCPSocket did not buffer, the OS would buffer a bit, then drop the packet, 
so sender's TCP algorithms would reduce the rate of sending letting the time 
for the application to process.  

This is my naïve interpretation of the problem and I might be wrong here.
But I think, indeed, the network and IO stack should do something against 
that.





More information about the Development mailing list