[Interest] QAbstractEventDispatcher - registerSocketNotifier(), unregisterSocketNotifier()

[Narolewski Jakub]

Hello :]

I had some time to do more tests experimenting with custom event
dispatchers.
To give more background, my use case is as follows:

1. There is one main thread - 'Server' - that currently contains
QWebSocketServer instance serving data to clients
2. There is one or more threads doing some additional work - 'Rooms'

Every client sends some data to server with frequency of 20Hz, by first
passing custom QEvent from Room thread to Server thread.
Server sends some data to every client with frequency of 20Hz.
Typically there are around 1-8 clients per server instance.

The goal was to check if I could 'speed it up' a bit by subclassing
QAbstractEventDispatcher and integrating libuv into my stack.
Some modifications were made to the sources of Qt:

1. As suggested by Thiago Macieira I have added to the
QAbstractEventDispatcher functions:

void enableSocketNotifier(noitifier)
void disableSocketNotifier(noitifier)

This allowed me to have finer control over when I want to destroy internal
liuv poll handlers and when I just want enable or disable them.

2. I have added small helper to the QSocketNotifier - a C-like, void*
handler to which I'm able to 'attach stuff'.
While not very pretty it allows me to completely remove parts of the code
that are responsible for bookkeeping relations between sockets and libuv's
pollers.

VTune data from my 'production-like' server [ 1x2GHz vCore, 1GB RAM on OVH
cluster ] is here:
qt_native:
https://drive.google.com/open?id=1mpRVEcuZWr9YCibXkl_acqjtJPhstept

libuv:
https://drive.google.com/open?id=1YEih47QZHHg0V8N90moSxjU7Diu4qLgm

Generally it is looking nice. I was able to shave 15% - 20% total CPU time
on Server thread just by swapping dispatchers.
For some odd reason Room thread is now about 140% slower - but hey, never
said it was bug free solution :P

Thiago Macieira wrote:

> The one thing to look after is that QAbstractSocket has a shortcut to
skip
> some of the signal-slot mechanisms of QSocketNotifier and be notified
> immediately by way of a virtual. So there's a second place to deal with
> registering/unregistering, aside from QSocketNotifier.

For the love of God I can't find it. Do I understand correctly, that I
should look for another place that call register / unregister
SocketNotifier() directly?

niedz., 19 sty 2020 o 17:57 Narolewski Jakub <izowiuz at gmail.com> napisał(a):

> I finally had some time to really look at this and even very basic
> implementation works great in my use case.
> Adding just functions enabling and disabling socket notifiers allows me to
> have finer control over when the stuff on my end gets deleted.
> There are still some crashes that I have to hunt down. Mostly around of
> line 'QObjectPrivate *d = receiver->d_func()' from:
>
> // Qt code -- >
>
> // qcoreapplication.cpp, from line 1083 as of Qt 5.14
>
> // Qt enforces the rule that events can only be sent to objects in
> // the current thread, so receiver->d_func()->threadData is
> // equivalent to QThreadData::current(), just without the function
> // call overhead.
> QObjectPrivate *d = receiver->d_func();
> QThreadData *threadData = d->threadData;
> QScopedScopeLevelCounter scopeLevelCounter(threadData);
> if (!selfRequired)
>     return doNotify(receiver, event);
> return self->notify(receiver, event);
>
> // < -- Qt code
>
> but it might be just some error on my end.
>
> One more question I have is about potential optimizations concerning
> bookkeeping relations between QSocketNotifier and 'foreign' part on the
> side of QAbstractEventDispatcher.
> From what I saw, currently subclasses of QAED tend to keep relations
> between socket notifiers and some 'system' counterpart in indexes - QHashes
> for example.
> Do you think it would be possible to implement an interface allowing
> client to inject some additional information directly into the
> QSocketNotifier to skip 'finding' it in QAED?
> Maybe something similar to 'std::any attachedData' directly in
> QSocketNotifier?
> QSocketNotifier being QObject can of course be \'setPropertied\' to hold
> some external data but, to be honest, I never actually benchmarked
> performance of this.
>
>
> pon., 30 gru 2019 o 22:00 Thiago Macieira <thiago.macieira at intel.com>
> napisał(a):
>
>> On Monday, 30 December 2019 15:31:14 -03 Narolewski Jakub wrote:
>> > I did not notice if unregisterTimer() and registerTimer() are also
>> called
>> > so often but their non-destructive switching could also come in handy.
>> > Do you think that changing current QSocketNotifier::setEnabled( bool )
>> > behaviour to actually enable / disable notifiers is a good idea?
>>
>> I don't see why not. In fact, I'm positive it is a good idea. In all
>> implementations, there's a cost associated with registering a notifier.
>> So the
>> worst case scenario is that in an implementation it's impossible to
>> disable,
>> forcing us to fall back to deregistering with the low-level
>> implementation.
>> That's exactly what we do today, so it's no worse. We do need some
>> bookkeeing
>> so we can re-register on setSocketDescriptorEnabled(), but I guess that
>> it's
>> the bookkeeping that's already there.
>>
>> The drawback is of the increased memory consumption by QSocketNotifiers
>> that
>> stay mostly disabled. That is the case for the write notifiers for all
>> sockets, which are only enabled when they have something to write. I
>> think
>> that's an acceptable trade-off.
>>
>> > I don't know much about Qt at this level or how much of current code
>> > depends on them actually registering and deregistering.
>> >
>> > Also, does QSocketNotifier 'know' when it is supposed to release
>> underlying
>> > socket?
>>
>> Today, on setEnabled(false) or destruction. If we implement the new
>> feature,
>> only on destruction.
>>
>> > Could registering and deregistering be moved entirely to its constructor
>> > and destructor?
>>
>> That's what I hope.
>>
>> > I don't know how QAbstractEventDispatcher interops with QTimer, I will
>> try
>> > to research it a bit more.
>>
>> It's basically the same.
>>
>> The one thing to look after is that QAbstractSocket has a shortcut to
>> skip
>> some of the signal-slot mechanisms of QSocketNotifier and be notified
>> immediately by way of a virtual. So there's a second place to deal with
>> registering/unregistering, aside from QSocketNotifier.
>>
>> --
>> Thiago Macieira - thiago.macieira (AT) intel.com
>>   Software Architect - Intel System Software Products
>>
>>
>>
>> _______________________________________________
>> Interest mailing list
>> Interest at qt-project.org
>> https://lists.qt-project.org/listinfo/interest
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/interest/attachments/20200203/5d75d966/attachment-0001.html>