[Qtwebengine] Revisiting texture streaming

[Mansoor C]

Thanks Jocelyn for your detailed reply. I had assumed, probably
incorrectly, that if Chromium is using the same texture but only
uploading a new image to it, it won't change its resource ID. If it
sends a new resource ID every time, I can see how the synchronization
between Chromium GPU and Qt Renderer threads work. I will collect some
logs to confirm this.

StreamVideoNode behavior should be the same as YUVVideoNode as far as
how chromium sends over associated resources. I will double check.

--Mansoor

>
> On Tue, Mar 11, 2014 at 04:09:32PM -0500, Mansoor C wrote:
>> It is my understanding that when two GL contexts (threads) share
>> resources, it is up to the application to ensure that whenever a
>> resource (texture in this case) is being modified, it is not accessed
>> by other threads. For non-stream textures, I see this constraint is
>> met in the way qt-renderer thread interacts with chromium-gpu thread.
>> I see that DFN::preprocess blocks qt-renderer thread whenever a new
>> MailboxTexture is received from chromium whose textureId is  not set
>> yet. Then chromium gpu thread unblocks qt-renderer after all
>> MailboxTextures are ready. My understanding is that this happens once
>> for each new texture. This is fine for regular textures because once
>> an image is loaded on a texture by gpu thread, it does not ever has to
>> change again. If content changes later, a different texture will be
>> sent by chromium and previous texture will be removed.
>
> There a subtle assumption in the code that isn't written down so well in comments. The mailbox wait and fetch doesn't happen only for each new texture, but rather for each new *resource*. A new resource id can identify both a new texture, or the new state of an existing texture ID.
>
> That a texture ID is the same as a previous one is something that is abstracted away by the resources and mailboxes system. We get a resource ID, we fetch the mailbox (waiting for its associated sync point) and the fact that this ends up giving us a GL texture ID is kind of an implementation detail, or a detail of a lower level layer of abstraction to put it differently.
>
> You're absolutely right about the threads, we have to make sure externally that a texture isn't used by one thread before we let any other thread write to it. In our case it's the sync point waiting that makes sure that we wait until it's ready to display it (since the Chromium GPU thread might still be in the process of rendering/uploading to this texture), and on the other side it's the resourcesToRelease list that takes care of telling Chromium that a texture ID isn't in the graphic pipeline anymore and that it can be returned to the pool.
>
>> But what about video textures? I think that a MailboxTexture
>> containing video stays unchanged in the RenderPass (its textureId
>> stays the same) - only a new video frame will be periodically uploaded
>> to that texture by the gpu thread. This implies that the qt-renderer
>> thread would never wait on this texture to be updated - except for the
>> very first time when this texture appeared new in a RenderPass. Could
>> that be a problem given that the gpu thread would be updating this
>> texture while at the same time qt-renderer thread might be drawing
>> with the same texture?
>
> From my experience, each new frame of a video texture will be provided through a new resource ID. This allows both the delegat*ing* renderer in the render process to know when a texture is still used by the final rendering (until this resource ID has been returned to the pool through the cc::CompositorFrameAck) from which it knows if it can send the new frame using the same texture ID, or a new one from the pool of textures.
>
>> I don't know enough about GPU shared contexts but from what I
>> understood this could be a potential problem. Could it cause any
>> rendering artifacts? This is not just about the recent StreamVideo
>> material that my patch introduced in qtwebengine - but I think same
>> thing would happen in case of YUVVideo material as well. I haven't
>> studied YUVVideo code but I assume that the same three MailboxTextures
>> would be reused over and over again while video is playing.
>
> From my understanding, the key points for the YUVVideo synchronisation are:
> - Each frame of each channel gets a new resource ID
> - Each of those resources come with a sync point that is only released when the Chromium GPU thread executed the required command stream to make it ready for rendering. This is ultimately triggered by SyncPointManager::RetireSyncPoint, which I believe is called by context3d->insertSyncPoint() in ResourceProvider::PrepareSendToParent when the render process is ready to send the frame with the bundled video frame to the parent compositor (our RenderWidgetHostViewQt).
>
> I've been grepping around a bit and it seems like VideoLayerImpl::WillDraw is the one triggering the new resource id for each frame by calling resource_provider->CreateResourceFromTextureMailbox. I don't know what could be different for stream texture resources (they seem to be handled the same way as YUV video frames in the code), but from what I can see they should also get a new resource ID for each frame and this should trigger proper synchronization from our side.
>
> Cheers,
> Jocelyn