[Development] QProcess fork() failure and overcommit

[Simon Hausmann]

Hi,


It would be interesting to see /proc/<pid>/maps of the KDevelop process when fork fails. I would imagine - like Christian - that there is a shortage of memory mappings. It's odd that this is an issue, as there shouldn't be big memory mappings _usually_. But for example if KDevelop ends up using QtScript and you're on a 64-bit system, then we end up allocating 2GB of address space, which the kernel has to copy (in terms of page tables) when forking. It could be that you're short of that.


We should probably mark these address spaces with MADV_DONTFORK.


But can you check first if KDevelop ends up using QtScript by chance?


Simon

________________________________
From: Development <development-bounces+simon.hausmann=qt.io at qt-project.org> on behalf of René J.V. Bertin <rjvbertin at gmail.com>
Sent: Tuesday, March 7, 2017 2:04:58 PM
To: development at qt-project.org
Subject: [Development] QProcess fork() failure and overcommit

Hi,

I have a bit of an intriguing issue I hope someone here could help me understand. If not, sorry for the noise.

I'm seeing occasional QProcess failures where QProcess:waitForStarted() fails and gives rise to errors like

kdevplatform.vcs: "DVCSJob::start: git ls-files -- kclock.cpp failed to start: Resource error (fork failure): Cannot allocate memory"

Those come and go in episodes; restarting the affected application always helped too (for a while).

Last time this happened htop reported I had 6955Mb used of 7899Mb, and not even 10% swap used (1151Mb of 16Gb) but despite that the issue went away when I turned overcommit back on (I usually run with overcommit_memory=2 and overcommit_ratio=80).

This only happened to me with KDevelop5 until now, i.e. a Qt5/KF5 based application, running on a KDE4 (= Qt4-based) desktop.
If memory contention were really the cause here I would expect to see traces of similar failures elsewhere too because KDevelop isn't exactly the only KDE application that makes generous use of QProcess.

Is there anything in QProces (Qt5) vs. the Qt4 version that could explain fork() failing, or else can it be the way QProcess is being used which causes this in KDevelop but not other applications?

FWIW, not only do I see this in KDevelop exclusively, I've also seen it only when executing git commands through QProcess.

I've been seeing this for several months now, meaning with Qt 5.6.2, 5.7.1 and now 5.8.0 and kernels 4.7, 4.9 and possibly 4.5.7 (all with Con Kolivas's patches).

Thanks,
René
_______________________________________________
Development mailing list
Development at qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20170307/d817c11f/attachment.html>