[Development] Qt/Quick and VNC [ was: damage tracking ... ]

[Uwe Rathmann]

Hi all,

sorry for hijacking a thread in a slightly different direction, but as
I was working on a VNC solution for Qt/Quick for EGLFS recently.

My VNC server ( BSD-3-Clause license ) can be found here:
https://github.com/uwerat/vnc-eglfs

The main reason why I'm not happy with the official VNC plugin is that
it does not support any type of OpenGL. Beside the known limitations of
the software backend this means that custom scene graph nodes usually do
not work at all.

My customer also needed a solution, where the application is displayed
locally and remotely at the same time.

> [ Giuseppe D'Angelo wrote: ]
> Properly signalling damage areas has a lot 
> of interesting use cases, from compositors to efficiently enable 
> remoting solutions (frame streaming/VNC/etc.).

Well, Qt/Quick is usually for modern user interfaces with smooth state
transitions, what means you have to deal with a lot of fullscreen
operations like fading in/out of pages etc. that have to run at 60fps.

While limiting update regions to the damage areas is of course often an
improvement - it is is no game changer as you also have the use cases
above. IMHO the solution to the problem can only be: using image
compression formats like JPEG or even better H.264.

The current state of my VNC server implementation supports JPEG
using the encoder ( libjpeg-turbo ) that comes with Qt.
But even with having an expensive glReadPixels call and JPEG compression
for each frame I was able to display ~20fps remotely for a 1200x800
window. ( numbers reported from xtigervncviewer -Log '*:stderr:100' )

Unfortunately I didn't have enough time for the project and was not
able to get the libva running. This should avoid the bottlenecks above
by encoding on the GPU before downloading the compressed frame.
( Will try to do it at a later point in time)

If someone is more familiar with the VA_API and is interested in my
approach - please let me know ?

A comment on the VNC platfom plugin that comes with Qt: the approach
that has been taken has its pros ( also works for widgets ) and cons (
see above ). But I noticed that there are a couple of features that are
missing that could be added easily:

- more Encodings ( f.e JPEG )
- Mouse wheel support ( see my code )
- more key codes ...

Looks a bit like the code had been adapted from 15 years old QWS code
without checking what has happened in the mean time:
https://github.com/rfbproto/rfbproto/blob/master/rfbproto.rst

cu,
Uwe