[Development] Qt/Quick and VNC

[Uwe Rathmann]

On 6/4/22 23:20, Giuseppe D'Angelo via Development wrote:

> I'd tend to agree; for some of our customers we indeed used H.264 
> encoding using VDPAU/VAAPI.

Actually H.264 covers the idea of sending differences between 2 frames
only - but without requiring any support of Qt/Quick.

However you will have a hard time finding a VNC viewer supporting H.264.
IIRC xtigervnc added it very recently - never heard about any other
viewer doing the same so far ( that's why I started with JPEG )

But in most VNC scenarios the user only needs to have one combination
that works and when the device running the Qt/Quick application has
VDPAU/VAAPI support then having the H.264 option IMHO offers the best
solution possible.

> However, for lower end embedded platforms lacking HW video encoders 
> (but still with a GPU), we had to do CPU encoding of the changes and 
> having damage areas helps tremendously.

Results depend on the device, the size of the frames and its content (
being more quiet equals to better compression ) - but at least these are
some numbers from one of my test setups:

     - Intel NUC with on board GPU
     - window of 800x800 pixels
     - a couple of indeterminated progress bars ( using qskinny ):

xtigervnc reports:

    - using raw encoding: ~2fps
    - using Tight/JPEG: ~20fp

I would argue, that image compression offers a "working" solution, while
knowing the update regions only is an improvement, but remains on the
"non working" side for modern UIs.

But of course the RFB protocol supports both type of optimizations and
in case the update region is known a VNC server should only send them.

Concerning the image compression:

In the beginning the viewer sends all encodings (
https://github.com/rfbproto/rfbproto/blob/master/rfbproto.rst#encodings
) it supports to the server. Then the server makes its decision how to
send the image data for the dirty rectangles ( might be even possible to
use different encodings for each rect ).

However the VNC server that is implemented in the VNC platform plugin
completely ignores the requests/capabilities from the viewer and
supports the mandatory Raw format (
https://github.com/rfbproto/rfbproto/blob/master/rfbproto.rst#raw-encoding
) only.

IMHO this is simply not good enough to achieve satisfying results for
modern UIs.

Uwe