[Development] 回复: Qt/Quick and VNC

[Zhang JiDe]

The mention of VNC in damage tracking is just one example, since the frame is encoded, it removes the duplication between the two frames, so I'm not sure how much help by providing damage tracking for VNC, but that once know the damaged area, it will be possible to reduce the amount of data read and encoded, it's beneficial.

Let's go back to talking about damage tracking, I think we can do the following for it:

1. Add an interface such as setSourceDamageArea(const QList<QRectF>) to the node that renders the material such as QSGImageNode, allowing to provide it with the damaged area of ​​the source material in updatePaintNode
2. Mark all dirty nodes. When implementing specific rendering operations in the QSGRenderer-derived class, calculate the area where all the dirty nodes are maped to the rendering target (such as the native surface of the window). After the rendering is completed, the calculation result is regarded as the damaged area, and passed to the native window system.
4. Support eglSwapBuffersWithDamageKHR and other interfaces
5. qtwayland adapts damage tracking

The above does not mention clipping before rendering, because I think it is difficult for us to know the maximum range of dirty areas before rendering, but after rendering it is easier for us to know the dirty areas, which is we can do, it's will help us a lot of.

________________________________
发件人: Development <development-bounces at qt-project.org> 代表 Uwe Rathmann <Uwe.Rathmann at tigertal.de>
发送时间: 2022年6月5日 7:55
收件人: development at qt-project.org <development at qt-project.org>
主题: Re: [Development] Qt/Quick and VNC

On 6/4/22 23:20, Giuseppe D'Angelo via Development wrote:

> I'd tend to agree; for some of our customers we indeed used H.264
> encoding using VDPAU/VAAPI.

Actually H.264 covers the idea of sending differences between 2 frames
only - but without requiring any support of Qt/Quick.

However you will have a hard time finding a VNC viewer supporting H.264.
IIRC xtigervnc added it very recently - never heard about any other
viewer doing the same so far ( that's why I started with JPEG )

But in most VNC scenarios the user only needs to have one combination
that works and when the device running the Qt/Quick application has
VDPAU/VAAPI support then having the H.264 option IMHO offers the best
solution possible.

> However, for lower end embedded platforms lacking HW video encoders
> (but still with a GPU), we had to do CPU encoding of the changes and
> having damage areas helps tremendously.

Results depend on the device, the size of the frames and its content (
being more quiet equals to better compression ) - but at least these are
some numbers from one of my test setups:

     - Intel NUC with on board GPU
     - window of 800x800 pixels
     - a couple of indeterminated progress bars ( using qskinny ):

xtigervnc reports:

    - using raw encoding: ~2fps
    - using Tight/JPEG: ~20fp

I would argue, that image compression offers a "working" solution, while
knowing the update regions only is an improvement, but remains on the
"non working" side for modern UIs.

But of course the RFB protocol supports both type of optimizations and
in case the update region is known a VNC server should only send them.

Concerning the image compression:

In the beginning the viewer sends all encodings (
https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Frfbproto%2Frfbproto%2Fblob%2Fmaster%2Frfbproto.rst%23encodings&data=05%7C01%7C%7Ce7bcbf08917848282cb408da46c8fada%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637900126255153793%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ikC7697jfBumGexe6HyKIPnX5JlDLz3n2Ac9PaaMigk%3D&reserved=0
) it supports to the server. Then the server makes its decision how to
send the image data for the dirty rectangles ( might be even possible to
use different encodings for each rect ).

However the VNC server that is implemented in the VNC platform plugin
completely ignores the requests/capabilities from the viewer and
supports the mandatory Raw format (
https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Frfbproto%2Frfbproto%2Fblob%2Fmaster%2Frfbproto.rst%23raw-encoding&data=05%7C01%7C%7Ce7bcbf08917848282cb408da46c8fada%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637900126255153793%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MyXR6tdEPXl9fZHf2Ma4H3S8QP66y%2B742%2BeBldrI65o%3D&reserved=0
) only.

IMHO this is simply not good enough to achieve satisfying results for
modern UIs.

Uwe



_______________________________________________
Development mailing list
Development at qt-project.org
https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.qt-project.org%2Flistinfo%2Fdevelopment&data=05%7C01%7C%7Ce7bcbf08917848282cb408da46c8fada%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637900126255153793%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=i79lj0o0jkX%2FqKVGrkZMlZa6MhsSwS7i7BknCQbUsWo%3D&reserved=0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20220606/337839eb/attachment.htm>