[Interest] QAbstractEventDispatcher - registerSocketNotifier(), unregisterSocketNotifier()

[Narolewski Jakub]

Just fixing small typo in my last email.

"Typically there are around 108 clients per server instance"

pon., 3 lut 2020 o 21:37 Narolewski Jakub <izowiuz at gmail.com> napisał(a):

> Hello :]
>
> I had some time to do more tests experimenting with custom event
> dispatchers.
> To give more background, my use case is as follows:
>
> 1. There is one main thread - 'Server' - that currently contains
> QWebSocketServer instance serving data to clients
> 2. There is one or more threads doing some additional work - 'Rooms'
>
> Every client sends some data to server with frequency of 20Hz, by first
> passing custom QEvent from Room thread to Server thread.
> Server sends some data to every client with frequency of 20Hz.
> Typically there are around 1-8 clients per server instance.
>
> The goal was to check if I could 'speed it up' a bit by subclassing
> QAbstractEventDispatcher and integrating libuv into my stack.
> Some modifications were made to the sources of Qt:
>
> 1. As suggested by Thiago Macieira I have added to the
> QAbstractEventDispatcher functions:
>
> void enableSocketNotifier(noitifier)
> void disableSocketNotifier(noitifier)
>
> This allowed me to have finer control over when I want to destroy internal
> liuv poll handlers and when I just want enable or disable them.
>
> 2. I have added small helper to the QSocketNotifier - a C-like, void*
> handler to which I'm able to 'attach stuff'.
> While not very pretty it allows me to completely remove parts of the code
> that are responsible for bookkeeping relations between sockets and libuv's
> pollers.
>
> VTune data from my 'production-like' server [ 1x2GHz vCore, 1GB RAM on OVH
> cluster ] is here:
> qt_native:
> https://drive.google.com/open?id=1mpRVEcuZWr9YCibXkl_acqjtJPhstept
>
> libuv:
> https://drive.google.com/open?id=1YEih47QZHHg0V8N90moSxjU7Diu4qLgm
>
> Generally it is looking nice. I was able to shave 15% - 20% total CPU time
> on Server thread just by swapping dispatchers.
> For some odd reason Room thread is now about 140% slower - but hey, never
> said it was bug free solution :P
>
> Thiago Macieira wrote:
>
> > The one thing to look after is that QAbstractSocket has a shortcut to
> skip
> > some of the signal-slot mechanisms of QSocketNotifier and be notified
> > immediately by way of a virtual. So there's a second place to deal with
> > registering/unregistering, aside from QSocketNotifier.
>
> For the love of God I can't find it. Do I understand correctly, that I
> should look for another place that call register / unregister
> SocketNotifier() directly?
>
> niedz., 19 sty 2020 o 17:57 Narolewski Jakub <izowiuz at gmail.com>
> napisał(a):
>
>> I finally had some time to really look at this and even very basic
>> implementation works great in my use case.
>> Adding just functions enabling and disabling socket notifiers allows me
>> to have finer control over when the stuff on my end gets deleted.
>> There are still some crashes that I have to hunt down. Mostly around of
>> line 'QObjectPrivate *d = receiver->d_func()' from:
>>
>> // Qt code -- >
>>
>> // qcoreapplication.cpp, from line 1083 as of Qt 5.14
>>
>> // Qt enforces the rule that events can only be sent to objects in
>> // the current thread, so receiver->d_func()->threadData is
>> // equivalent to QThreadData::current(), just without the function
>> // call overhead.
>> QObjectPrivate *d = receiver->d_func();
>> QThreadData *threadData = d->threadData;
>> QScopedScopeLevelCounter scopeLevelCounter(threadData);
>> if (!selfRequired)
>>     return doNotify(receiver, event);
>> return self->notify(receiver, event);
>>
>> // < -- Qt code
>>
>> but it might be just some error on my end.
>>
>> One more question I have is about potential optimizations concerning
>> bookkeeping relations between QSocketNotifier and 'foreign' part on the
>> side of QAbstractEventDispatcher.
>> From what I saw, currently subclasses of QAED tend to keep relations
>> between socket notifiers and some 'system' counterpart in indexes - QHashes
>> for example.
>> Do you think it would be possible to implement an interface allowing
>> client to inject some additional information directly into the
>> QSocketNotifier to skip 'finding' it in QAED?
>> Maybe something similar to 'std::any attachedData' directly in
>> QSocketNotifier?
>> QSocketNotifier being QObject can of course be \'setPropertied\' to hold
>> some external data but, to be honest, I never actually benchmarked
>> performance of this.
>>
>>
>> pon., 30 gru 2019 o 22:00 Thiago Macieira <thiago.macieira at intel.com>
>> napisał(a):
>>
>>> On Monday, 30 December 2019 15:31:14 -03 Narolewski Jakub wrote:
>>> > I did not notice if unregisterTimer() and registerTimer() are also
>>> called
>>> > so often but their non-destructive switching could also come in handy.
>>> > Do you think that changing current QSocketNotifier::setEnabled( bool )
>>> > behaviour to actually enable / disable notifiers is a good idea?
>>>
>>> I don't see why not. In fact, I'm positive it is a good idea. In all
>>> implementations, there's a cost associated with registering a notifier.
>>> So the
>>> worst case scenario is that in an implementation it's impossible to
>>> disable,
>>> forcing us to fall back to deregistering with the low-level
>>> implementation.
>>> That's exactly what we do today, so it's no worse. We do need some
>>> bookkeeing
>>> so we can re-register on setSocketDescriptorEnabled(), but I guess that
>>> it's
>>> the bookkeeping that's already there.
>>>
>>> The drawback is of the increased memory consumption by QSocketNotifiers
>>> that
>>> stay mostly disabled. That is the case for the write notifiers for all
>>> sockets, which are only enabled when they have something to write. I
>>> think
>>> that's an acceptable trade-off.
>>>
>>> > I don't know much about Qt at this level or how much of current code
>>> > depends on them actually registering and deregistering.
>>> >
>>> > Also, does QSocketNotifier 'know' when it is supposed to release
>>> underlying
>>> > socket?
>>>
>>> Today, on setEnabled(false) or destruction. If we implement the new
>>> feature,
>>> only on destruction.
>>>
>>> > Could registering and deregistering be moved entirely to its
>>> constructor
>>> > and destructor?
>>>
>>> That's what I hope.
>>>
>>> > I don't know how QAbstractEventDispatcher interops with QTimer, I will
>>> try
>>> > to research it a bit more.
>>>
>>> It's basically the same.
>>>
>>> The one thing to look after is that QAbstractSocket has a shortcut to
>>> skip
>>> some of the signal-slot mechanisms of QSocketNotifier and be notified
>>> immediately by way of a virtual. So there's a second place to deal with
>>> registering/unregistering, aside from QSocketNotifier.
>>>
>>> --
>>> Thiago Macieira - thiago.macieira (AT) intel.com
>>>   Software Architect - Intel System Software Products
>>>
>>>
>>>
>>> _______________________________________________
>>> Interest mailing list
>>> Interest at qt-project.org
>>> https://lists.qt-project.org/listinfo/interest
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/interest/attachments/20200203/1042780b/attachment-0001.html>