[Development] A QtCore class for event-driven jobs

[Konstantin Ritt]

Simply can not ignore this thread :)
Like Andre, I have written my own jobs implementation, too.
And I completely agree that not having some kind of job manager makes QJob
API of quite limited usefulness (and like David mentioned earlier,
QNetworkReply is a QNAM's "job", where QNAM is indeed a manager).
Plz don't forget that some generic API in Qt, i.e. like async jobs, should
be flexible enough to be used for quite different tasks by variety of
users, not only just to make a "standard base" in future KIO implementation.

Namely, instead of
[code]
QJob* job = someoperation(some parameters);
connect(job, SIGNAL(result(QJob*)),
this, SLOT(handleResult(QJob*)));
job->start();
[/code]

one would probably prefer

[code]
job =  someoperation(some parameters);
manager->enqueue(job);
[/code]
, where "someoperation" could be manager's registered job factory or QJob
sub-class, etc.

This way, job not necessarily should derive from QObject; all required
signals and methods could be provided by some kind of job watcher [or job
sequence watcher] or even by the manager itself.
Moreover, in case where one needs i.e. to visualize jobs sequence execution
progress with just a single progressbar, having to connect to every job's
progress/finished/error/whatever signals and/or suspend/resume/abort slots
once the job is started makes the implementation not really convenient to
the user. Connecting to the manager just once looks like a better option
for a generic case, IMO.

Another major possible obstacle is a need to suspend job execution up until
some additional info has taken from the user (i.e. proxy authorization,
non-critical errors handling/reaction, etc.).
In common case, this requires appearing some dialog in GUI thread and thus
a making blocking call to some user-defined callback, if any (and possibly
job re-execution after a successive call,  e.g. "re-try with admin
privileges", where re-enqueuing is definitely not an option). There could
be more elegant solution but still, implementation must not force the user
to re-invent a wheel every time he needs it.

Kind regards,
Konstantin


2013/9/6 Andre Somers <andre at familiesomers.nl>

>  Op 6-9-2013 19:56, Matt Broadstone schreef:
>
> On Fri, Sep 6, 2013 at 1:52 PM, David Faure <david.faure at kdab.com> wrote:
>
>> I would like to propose the inclusion of a QJob class in QtCore, based on
>> years of experience with KIO::Job and KJob.
>>
>> The idea is to encapsulate an event driven asynchronous operation into a
>> job
>> class. Example use cases:
>> - a network download with QNetworkAccessManager.
>> - operations (command+reply) over a QLocalSocket or QTcpSocket (like
>> akonadi).
>> - long async dbus calls (special case of the previous line)
>> - long tasks executed by external processes (e.g. "make" from an IDE,
>> "unrar"
>> from an archive program)
>> ...
>>
>> At the core, QJob is really just a class with start() and kill(), calling
>> pure
>> virtual methods doStart() and doKill(), and signals, most importantly the
>> result(QJob *) signal.
>>
>> The expected use case is:
>>
>>  void SomeClass::methodWithAsynchronousJobCall()
>>  {
>>    QJob* job = someoperation(some parameters);
>>    connect(job, SIGNAL(result(QJob*)),
>>            this, SLOT(handleResult(QJob*)));
>>    job->start(); // or it could just autostart, which I actually prefer...
>>  }
>>    (other connects, specific to the job)
>>
>>  And handleResult is usually at least:
>>
>>  void SomeClass::handleResult( QJob *job )
>>  {
>>    if (job->error()) {
>>        // handle error
>>    } else {
>>       // handle succesful job completion, if needed
>>    }
>>  }
>>
>> But it can and should also have the following features:
>> * error code, error text
>> * suspend/resume with doSuspend/doResume virtual methods
>> * capabilities Killable and Suspendable, to avoid trying these on jobs
>> that
>> don't support them
>> * kill(Quietly) vs kill(EmitResult), for the app's convenience
>> * a finished signal that is emitted with both types of killing, for things
>> like progress dialogs
>> * auto-deletion (on by default, can be turned off)
>> * synchronous exec() using a QEventLoop, with a big fat huge warning
>> about not
>> using that in GUI apps (ideally only to be used in unittests or separate
>> threads).
>> * more standard signals for progress info, messages, warnings..
>>
>> The whole point of standardizing such signals is that it allows generic
>> GUIs
>> to be built on top, so that your app or your desktop can show the
>> progress of
>> multiple concurrent jobs, of different types (file download, CD burning,
>> mail
>> checking, etc. etc.)
>>
>> Finally, for the benefit of job implementors, QJob would support sub-jobs.
>> The job implementation would choose when to create these subjobs (all at
>> once
>> initially, to have them run in parallel, or one after the other, for
>> sequence
>> of operations). KDE currently does that in a subclass (KCompositeJob) but
>> Thiago and I (and kio, and akonadi) agree that it's better to have it all
>> in
>> one class instead.
>>
>> We also have a standard interface for error handling so that all jobs
>> from a
>> given framework can have their error handled the same way, but thinking
>> about
>> it, that part could stay separate, at least for now.
>>
>> Well, that's it. So why this email? Because Thiago asked me to, and to
>> gather
>> some support. I plan to make a merge request for Qt 5.2.
>>
>> Thiago asked more specifically:
>>
>> * API-wise, can't this be merged with QFuture?
>> -> no, because QFuture encapsulates a value, with blocking methods for
>> getting
>> the value, even available as casting-to-the-value. If we imagine a QFuture
>> that wraps a QJob, and the blocking method calling exec(), we'd have a lot
>> more nested event loops than is good for the health of our programs.
>> On the other hand, QJob would not be related to any value. It's really a
>> QObject that informs of progress via signals.
>>
>> * relation to QRunnable?
>> A runnable is also some sort of "job", but the implementation is
>> completely
>> different: a runnable is one sync method, while a qjob is all
>> signals/slots
>> based, with start() returning immediately. So one can't be used like the
>> other, a given task implementation is either a blocking task for a thread
>> with
>> no event loop (QRunnable) or an async task that can actually be used in
>> any
>> thread with an event loop.
>>
>> * relation to QNetworkReply?
>> If that one didn't exist yet, we'd definitely write it as a QJob
>> subclass. So
>> instead, I propose to wrap QNetworkReply into a QNetworkJob or something,
>> in
>> order to offer the QJob interface for QNAM requests. On one hand this
>> doesn't
>> have to go in at the same time as QJob itself, but OTOH it could be a
>> real-
>> world testcase for it, proving its usefulness and correctness...
>>
>> Any other questions?
>>
>>
>  +1, or more if possible :)
>
>  Is this code up somewhere already for early review? This is very much
> needed in Qt imho, I imagine it's one of those patterns that people roll
> themselves in many cases.
>
>  Matt
>
>
> First of all: I like the idea of a job API in Qt as well. Like many
> others, I have written my own before. However, I'd hope the design would
> not limit itself to just event-driven jobs, if I understand correctly what
> that means. Some of my jobs are just heavy processing that require
> multi-threading. Would this API suport that too, or would that be hacky
> again to implement? Another aspect that I build into my own solution and
> that I find valuable, is that I have added a mechanism to prevent the same
> work being done twice. If different parts of the code ask for the same
> calculation to be performed (that can happen in my case), they actually get
> a pointer to the same job back. That turns out to be very convenient. Note
> however, that that requires a job manager, and it doesn't look like that is
> in this design. In my case, I submit a request to the manager, who then
> returns a job.
>
> Note that in my case, I return QSharedPointer<Job>, so it is clear to
> everyone that the code that everyone that requests a job and gets one,
> can't treat it as if it was theirs exclusively.
>
> André
>
>
>
>
> _______________________________________________
> Development mailing list
> Development at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20130907/da71de87/attachment.html>