From mchishtie at gmail.com Tue Mar 11 22:09:32 2014 From: mchishtie at gmail.com (Mansoor C) Date: Tue, 11 Mar 2014 16:09:32 -0500 Subject: [Qtwebengine] Revisiting texture streaming Message-ID: Hi, Taking a second look at how delegated rendering works on qtwebengine, I wonder if there could be some shared-context related issues when rendering video ... It is my understanding that when two GL contexts (threads) share resources, it is up to the application to ensure that whenever a resource (texture in this case) is being modified, it is not accessed by other threads. For non-stream textures, I see this constraint is met in the way qt-renderer thread interacts with chromium-gpu thread. I see that DFN::preprocess blocks qt-renderer thread whenever a new MailboxTexture is received from chromium whose textureId is not set yet. Then chromium gpu thread unblocks qt-renderer after all MailboxTextures are ready. My understanding is that this happens once for each new texture. This is fine for regular textures because once an image is loaded on a texture by gpu thread, it does not ever has to change again. If content changes later, a different texture will be sent by chromium and previous texture will be removed. But what about video textures? I think that a MailboxTexture containing video stays unchanged in the RenderPass (its textureId stays the same) - only a new video frame will be periodically uploaded to that texture by the gpu thread. This implies that the qt-renderer thread would never wait on this texture to be updated - except for the very first time when this texture appeared new in a RenderPass. Could that be a problem given that the gpu thread would be updating this texture while at the same time qt-renderer thread might be drawing with the same texture? I don't know enough about GPU shared contexts but from what I understood this could be a potential problem. Could it cause any rendering artifacts? This is not just about the recent StreamVideo material that my patch introduced in qtwebengine - but I think same thing would happen in case of YUVVideo material as well. I haven't studied YUVVideo code but I assume that the same three MailboxTextures would be reused over and over again while video is playing. Is there something I am missing? Thanks, Mansoor Chishtie BlackBerry From jocelyn.turcotte at digia.com Wed Mar 12 11:57:35 2014 From: jocelyn.turcotte at digia.com (Jocelyn Turcotte) Date: Wed, 12 Mar 2014 11:57:35 +0100 Subject: [Qtwebengine] Revisiting texture streaming In-Reply-To: References: Message-ID: <20140312105735.GF13042@poutine.it.local> Hello, On Tue, Mar 11, 2014 at 04:09:32PM -0500, Mansoor C wrote: > It is my understanding that when two GL contexts (threads) share > resources, it is up to the application to ensure that whenever a > resource (texture in this case) is being modified, it is not accessed > by other threads. For non-stream textures, I see this constraint is > met in the way qt-renderer thread interacts with chromium-gpu thread. > I see that DFN::preprocess blocks qt-renderer thread whenever a new > MailboxTexture is received from chromium whose textureId is not set > yet. Then chromium gpu thread unblocks qt-renderer after all > MailboxTextures are ready. My understanding is that this happens once > for each new texture. This is fine for regular textures because once > an image is loaded on a texture by gpu thread, it does not ever has to > change again. If content changes later, a different texture will be > sent by chromium and previous texture will be removed. There a subtle assumption in the code that isn't written down so well in comments. The mailbox wait and fetch doesn't happen only for each new texture, but rather for each new *resource*. A new resource id can identify both a new texture, or the new state of an existing texture ID. That a texture ID is the same as a previous one is something that is abstracted away by the resources and mailboxes system. We get a resource ID, we fetch the mailbox (waiting for its associated sync point) and the fact that this ends up giving us a GL texture ID is kind of an implementation detail, or a detail of a lower level layer of abstraction to put it differently. You're absolutely right about the threads, we have to make sure externally that a texture isn't used by one thread before we let any other thread write to it. In our case it's the sync point waiting that makes sure that we wait until it's ready to display it (since the Chromium GPU thread might still be in the process of rendering/uploading to this texture), and on the other side it's the resourcesToRelease list that takes care of telling Chromium that a texture ID isn't in the graphic pipeline anymore and that it can be returned to the pool. > But what about video textures? I think that a MailboxTexture > containing video stays unchanged in the RenderPass (its textureId > stays the same) - only a new video frame will be periodically uploaded > to that texture by the gpu thread. This implies that the qt-renderer > thread would never wait on this texture to be updated - except for the > very first time when this texture appeared new in a RenderPass. Could > that be a problem given that the gpu thread would be updating this > texture while at the same time qt-renderer thread might be drawing > with the same texture? >From my experience, each new frame of a video texture will be provided through a new resource ID. This allows both the delegat*ing* renderer in the render process to know when a texture is still used by the final rendering (until this resource ID has been returned to the pool through the cc::CompositorFrameAck) from which it knows if it can send the new frame using the same texture ID, or a new one from the pool of textures. > I don't know enough about GPU shared contexts but from what I > understood this could be a potential problem. Could it cause any > rendering artifacts? This is not just about the recent StreamVideo > material that my patch introduced in qtwebengine - but I think same > thing would happen in case of YUVVideo material as well. I haven't > studied YUVVideo code but I assume that the same three MailboxTextures > would be reused over and over again while video is playing. >From my understanding, the key points for the YUVVideo synchronisation are: - Each frame of each channel gets a new resource ID - Each of those resources come with a sync point that is only released when the Chromium GPU thread executed the required command stream to make it ready for rendering. This is ultimately triggered by SyncPointManager::RetireSyncPoint, which I believe is called by context3d->insertSyncPoint() in ResourceProvider::PrepareSendToParent when the render process is ready to send the frame with the bundled video frame to the parent compositor (our RenderWidgetHostViewQt). I've been grepping around a bit and it seems like VideoLayerImpl::WillDraw is the one triggering the new resource id for each frame by calling resource_provider->CreateResourceFromTextureMailbox. I don't know what could be different for stream texture resources (they seem to be handled the same way as YUV video frames in the code), but from what I can see they should also get a new resource ID for each frame and this should trigger proper synchronization from our side. Cheers, Jocelyn From mchishtie at gmail.com Wed Mar 12 16:44:45 2014 From: mchishtie at gmail.com (Mansoor C) Date: Wed, 12 Mar 2014 10:44:45 -0500 Subject: [Qtwebengine] Revisiting texture streaming In-Reply-To: <20140312105735.GF13042@poutine.it.local> References: <20140312105735.GF13042@poutine.it.local> Message-ID: Thanks Jocelyn for your detailed reply. I had assumed, probably incorrectly, that if Chromium is using the same texture but only uploading a new image to it, it won't change its resource ID. If it sends a new resource ID every time, I can see how the synchronization between Chromium GPU and Qt Renderer threads work. I will collect some logs to confirm this. StreamVideoNode behavior should be the same as YUVVideoNode as far as how chromium sends over associated resources. I will double check. --Mansoor > > On Tue, Mar 11, 2014 at 04:09:32PM -0500, Mansoor C wrote: >> It is my understanding that when two GL contexts (threads) share >> resources, it is up to the application to ensure that whenever a >> resource (texture in this case) is being modified, it is not accessed >> by other threads. For non-stream textures, I see this constraint is >> met in the way qt-renderer thread interacts with chromium-gpu thread. >> I see that DFN::preprocess blocks qt-renderer thread whenever a new >> MailboxTexture is received from chromium whose textureId is not set >> yet. Then chromium gpu thread unblocks qt-renderer after all >> MailboxTextures are ready. My understanding is that this happens once >> for each new texture. This is fine for regular textures because once >> an image is loaded on a texture by gpu thread, it does not ever has to >> change again. If content changes later, a different texture will be >> sent by chromium and previous texture will be removed. > > There a subtle assumption in the code that isn't written down so well in comments. The mailbox wait and fetch doesn't happen only for each new texture, but rather for each new *resource*. A new resource id can identify both a new texture, or the new state of an existing texture ID. > > That a texture ID is the same as a previous one is something that is abstracted away by the resources and mailboxes system. We get a resource ID, we fetch the mailbox (waiting for its associated sync point) and the fact that this ends up giving us a GL texture ID is kind of an implementation detail, or a detail of a lower level layer of abstraction to put it differently. > > You're absolutely right about the threads, we have to make sure externally that a texture isn't used by one thread before we let any other thread write to it. In our case it's the sync point waiting that makes sure that we wait until it's ready to display it (since the Chromium GPU thread might still be in the process of rendering/uploading to this texture), and on the other side it's the resourcesToRelease list that takes care of telling Chromium that a texture ID isn't in the graphic pipeline anymore and that it can be returned to the pool. > >> But what about video textures? I think that a MailboxTexture >> containing video stays unchanged in the RenderPass (its textureId >> stays the same) - only a new video frame will be periodically uploaded >> to that texture by the gpu thread. This implies that the qt-renderer >> thread would never wait on this texture to be updated - except for the >> very first time when this texture appeared new in a RenderPass. Could >> that be a problem given that the gpu thread would be updating this >> texture while at the same time qt-renderer thread might be drawing >> with the same texture? > > From my experience, each new frame of a video texture will be provided through a new resource ID. This allows both the delegat*ing* renderer in the render process to know when a texture is still used by the final rendering (until this resource ID has been returned to the pool through the cc::CompositorFrameAck) from which it knows if it can send the new frame using the same texture ID, or a new one from the pool of textures. > >> I don't know enough about GPU shared contexts but from what I >> understood this could be a potential problem. Could it cause any >> rendering artifacts? This is not just about the recent StreamVideo >> material that my patch introduced in qtwebengine - but I think same >> thing would happen in case of YUVVideo material as well. I haven't >> studied YUVVideo code but I assume that the same three MailboxTextures >> would be reused over and over again while video is playing. > > From my understanding, the key points for the YUVVideo synchronisation are: > - Each frame of each channel gets a new resource ID > - Each of those resources come with a sync point that is only released when the Chromium GPU thread executed the required command stream to make it ready for rendering. This is ultimately triggered by SyncPointManager::RetireSyncPoint, which I believe is called by context3d->insertSyncPoint() in ResourceProvider::PrepareSendToParent when the render process is ready to send the frame with the bundled video frame to the parent compositor (our RenderWidgetHostViewQt). > > I've been grepping around a bit and it seems like VideoLayerImpl::WillDraw is the one triggering the new resource id for each frame by calling resource_provider->CreateResourceFromTextureMailbox. I don't know what could be different for stream texture resources (they seem to be handled the same way as YUV video frames in the code), but from what I can see they should also get a new resource ID for each frame and this should trigger proper synchronization from our side. > > Cheers, > Jocelyn From psz at dcclabs.tv Mon Mar 31 16:42:11 2014 From: psz at dcclabs.tv (Pawel Szlendak) Date: Mon, 31 Mar 2014 16:42:11 +0200 Subject: [Qtwebengine] QtWebEngine on embedded linux as of now Message-ID: <53397EC3.9060106@dcclabs.tv> Hi, I can see a config file for an embedded linux and was wondering if it is yet possible to build for embedded linux with this config? Best regards, Pawel From Zeno.Albisser at digia.com Mon Mar 31 18:08:46 2014 From: Zeno.Albisser at digia.com (Albisser Zeno) Date: Mon, 31 Mar 2014 16:08:46 +0000 Subject: [Qtwebengine] QtWebEngine on embedded linux as of now In-Reply-To: <53397EC3.9060106@dcclabs.tv> Message-ID: Hi, We are currently in the process of adding support for Qt Enterprise Embedded. It will be available to our customers soon. - Zeno On 3/31/14, 4:42 PM, "Pawel Szlendak" wrote: >Hi, > >I can see a config file for an embedded linux and was wondering if it is >yet possible to build for embedded linux with this config? > >Best regards, >Pawel >_______________________________________________ >QtWebEngine mailing list >QtWebEngine at qt-project.org >http://lists.qt-project.org/mailman/listinfo/qtwebengine