[Interest] ssue with QFutureWatcher cancel() delay

[Roland Hughes]

On 9/24/22 05:00, Sean Murphy wrote:
> I'm having an issue using the QtConcurrent module for processing large
> images, specifically allowing the user to cancel processing an image
> once it has begun.

Ahhh, welcome to the single-threadiness of Qt and one of the seemingly 
limitless faces of QTBUG-12055

https://bugreports.qt.io/browse/QTBUG-12055

Please let me take you on a short journey so you understand how you got 
screwed and perhaps you can find a way to unscrew yourself? I will steal 
a name from Dr. Who to hopefully make this more accessible to those who 
need it.

In the land of real computers with real operating systems we never had 
threads and never needed them. We had proper weighted processes and an 
OS accounting system that kept track of every byte of memory and 
resource allocated by every process. It kept track of all parents and 
children. Whenever you needed a "thread" to do something you created it 
via one of the many library routines and received its pid.

Acolytes of the Papal Mainframe were taught from birth that Gluttony was 
the deadliest of the Seven Deadly Sins. We never loaded big chunks or 
entire files of anything having any size into RAM and we certainly 
didn't create massive things in RAM. We always read/write reasonably 
sized chunks so every process is always yielding the Papal Processor to 
others while waiting for I/O. This yielding made the universe run 
smoother, allowed others to achieve enlightenment, and provided for a 
clean safe kosher kill via library routines or the command line

STOP/ID=pid

The OS would instantly kill the process, check the accounting system and 
free all resources allocated/locked by said pid.

That's what you are believing happens in the world of QFutureWatcher and 
it is not.

In the world of the x86-wanna-be-a-real-computer-one-day-when-I-grow-up 
world, the OS creators never figured out how to create a proper weight 
process. None of the operating systems created for this platform have a 
robust OS accounting system keeping track of its universe. Instead, what 
is a "process" on these platforms is too light to even be a thread on 
the real computers. Since they needed the ability to create other 
things, they invented threads which are lighter still and mostly outside 
of the realm of accounting. Threads can be left dangling when a parent 
is killed off due to the lack of accounting. Many times resources don't 
get freed when a thread is killed off for the very same reason.

In the x86 based Qt world you have the single main event loop where all 
GUI activity must occur. When you use the threading classes, especially 
with things like QFutureWatcher some portion of your thread activity 
will be at the mercy of the main event loop simply because some events 
will be queued there.

The whole QFuture and QThreadPool concept is great . . . if and only if 
you design your application from the mindset of a parent sending their 
kid off to college knowing full well they will only call when they need 
money. If you want them to live by your rules as they did under your 
roof, then you are going to have a rough time of it.

Too many people developing software on the x86 platform, even those 
writing its operating systems and libraries, never attended services at 
the Church of the Papal Mainframe. They never studied to be an acolyte 
and gluttony is the law of this land rather than being shunned for the 
deadliest of the deadly sins as it truly is.

Control returns to an event loop "at idle" or when you call 
processEvents(). Thiago hates people using that, but the design of Qt 
mandates the use of it, repeatedly, in otherwise good designs.

Those processing images tend to be the worst of the sinners. I've seen 
them read entire DVDs of content into RAM, then thread off a function to 
chew on that data and create yet another entire DVD of content in memory.

What does this do?

It creates the tightest loop you've ever seen. There is no yielding of 
the Papal Processor to others unless you get lucky the the dynamic 
memory allocation does it for you. There is no "idle" time for the event 
loop of your thread to process any events that got queued there. This

mLoadWatcher->disconnect();

never happened. It got queued on an event loop that never got processed. 
See QTBUG-12055. Likewise, this

mLoadWatcher->cancel();

got queued behind it. Your finished() signal was a direct connection, 
and it didn't know it wasn't supposed to happen so when the tightest 
loop in the world finished the function pointer assigned to execute when 
finished went ahead and executed.

Before you go thinking "Hey, I'll dig into the QFuture and QThreadPool 
classes and find out how to get a pid for the thread so I can kill it" 
don't. That is designed for things that run to completion and is a house 
of cards when they don't. Lots of things won't be deallocated/freed. The 
thread from the thread pool most likely will not be returned to the pool 
so you will exhaust the pool after just "a few" of these hard cancels.

**Ultimately you have to get rid of your tight loop.**

If you are using a library that has to have the entire image loaded into 
RAM or cannot have its tiling process pause for a processEvents() call 
then it is an improper library for your use case. If you can force said 
function/library to write the output to a file instead of physical RAM, 
you will create I/O yields that should allow your event loop to execute. 
Don't forget to nuke the file on cancel().

Make your thread into a stand-a-lone executable and get good with spawn.

https://linuxhint.com/posix-spawn-c-programming/

or exec

https://linuxhint.com/exec_linux_system_call_c/

Keep in mind that moving to a version of Qt that has the fix for 12055 
won't fix your problem. As far as I know cancel() is still a queued 
task. If it is changed from a queued task to a hard kill it is going to 
leave all kinds of dangling allocations and resources because none of 
the operating systems developed for the x86 have a proper OS accounting 
system. They cannot properly clean up behind a hard kill of a thread. 
They will do a better job (not perfect, but better) cleaning up behind a 
hard kill of an actual stand-a-lone PID created via spawn/exec.

I know that's not what you want to hear, but that's where you are at. No 
good way to kill a Glutton thread.

-- 
Roland Hughes, President
Logikal Solutions
(630)-205-1593

http://www.theminimumyouneedtoknow.com
http://www.infiniteexposure.net
http://www.johnsmith-book.com
http://www.logikalblog.com
http://www.interestingauthors.com/blog