[Development] [QtMultimedia] How to make to use a HW codecs by QML media player on Linux?

Wed May 25 14:07:58 CEST 2016

Now, I have recompiled the QtMM plugin (with default gstreamer 0.10), but
with adding of debug macro DEBUG_PLAYBIN.
Then I got following log: https://paste.kde.org/p6ztesnsq

Where I do not see any mentions about the selected gst sinks.. I see only
this:

{quote}
Set video output: QGstreamerVideoRenderer(0x153fe0)
Current sink: fakesink0 0x13e590 pending:  0x0 new sink: fakesink0 0x13e590
Video sink has not changed, skip video output reconfiguration
Video sink has chaged, reload video output
void QGstreamerPlayerSession::setVideoRenderer(QObject*)
Set video output: QGstreamerVideoRenderer(0x153fe0)
Current sink: fakesink0 0x13e590 pending:  0x0 new sink:
qvideosurfacegstsink0 0x1bc470
Reconfigure video output
The pipeline has not started yet, pending state: QMediaPlayer::StoppedState
Video sink has chaged, reload video output
void QGstreamerPlayerSession::setVideoRenderer(QObject*)
Set video output: QGstreamerVideoRenderer(0x153fe0)
Current sink: qvideosurfacegstsink0 0x1bc470 pending:  0x0 new sink:
qvideosurfacegstsink0 0x1bc470
Video sink has not changed, skip video output reconfiguration
{quote}

what is it 'fakesink', 'qvideosurfacegstsink0' ? where is any of desired:

{quote}
root at apalis-t30:~/Downloads# gst-inspect | grep sink | grep nv
omx:  nv_omx_audiosink: OpenMAX IL audiosink element
omx:  nv_gstbin_videosink: OpenMAX IL videosink element
omx:  nv_omx_videosink: OpenMAX IL videosink element
omx:  nv_omx_hdmi_videosink: OpenMAX IL hdmi videosink element
omx:  nv_omx_overlaysink: OpenMAX IL overlaysink element
omx:  nv_gl_eglimagesink: OpenMAX IL videosink element
nvxvimagesink:  nvxvimagesink: Video sink
root at apalis-t30:~/Downloads#
{quote}

sinks? I do not understand nothing.. o_O How I can see the selected sink
name?

2016-05-25 14:35 GMT+03:00 Denis Shienkov <denis.shienkov at gmail.com>:

> > BTW: As I remember, I saw some environment variable which allows to
> setup custom video sink... I need to see sources of QtMM... if I'm not
> mistaked..
> maybe it will help...
>
> I have found this env variables:
>
> * QT_GSTREAMER_WIDGET_VIDEOSINK
> * QT_GSTREAMER_WINDOW_VIDEOSINK
>
> and try to use it:
>
> $export QT_GSTREAMER_WIDGET_VIDEOSINK=nv_omx_hdmi_videosink
> $export QT_GSTREAMER_WINDOW_VIDEOSINK=nv_omx_hdmi_videosink
> $./my-qml-video-player
>
> but, this does not help...
>
> BR,
> Denis
>
>
>
> 2016-05-25 8:30 GMT+03:00 Denis Shienkov <denis.shienkov at gmail.com>:
>
>> > Gstreamer 1.0+ goes some way to addressing these issues....
>>
>> The Apalis T30 board has GPU from the NVidia. So, when I use
>> 'gst-inspect' I see some of 'nv' sinks.
>> So, as I understand, the easy ways to do it are:
>>
>> 1) I should try to use GStreamer 1.0 (instead of 0.10) "as is" with QtMM
>> (rebuild QtMM with this GStreamer), at first stage.
>>
>> 2) I need try to change QtMM sources and try to set the specific
>> vendor-video sinks there, at second stage... (in case the first stage does
>> not work).
>>
>> Is it?
>>
>> BTW: As I remember, I saw some environment variable which allows to setup
>> custom video sink... I need to see sources of QtMM... if I'm not mistaked..
>> maybe it will help...
>>
>> > but even within the standard packages there still seems to be multiple
>> APIs for codecs which produce EGL surfaces and none of them have been
>> utilized in QtMultimedia yet
>>
>> Could you please provide an examples of this API which is not adopted in
>> QtMM?
>>
>>
>>
>> BR,
>> Denis
>>
>>
>> 25.05.2016 4:44, Andrew den Exter пишет:
>>
>> The short answer is some amount of platform adaptation will be required
>> to enable hardware acceleration on your board.  Looking at the page you
>> linked to; that adaptation might be achieved by creating an implementation
>> of QGstBufferPoolPlugin (gstreamer 0.10 only) which exposes EGL surfaces
>> associated with the x-nv-yuv buffers produced by the accelerated codec.
>> Or you might also need to create your own video node by implementing QSGVideoNodeFactoryPlugin or
>> something more invasive may be required, it really depends on how the
>> vendor implemented their gstreamer plugins
>>
>> The reason hardware acceleration doesn't just work is that hardware
>> vendors have traditionally had to create extensions to gstreamer in order
>> to implement their hardware accelerated plugins, and these extensions tend
>> to require a vendor specific video sink to work.  If all you care about is
>> rendering to the full dimensions of a native window then autovideosink will
>> generally resolve to that vendor specific video sink and that's why the
>> standard example command line examples tend to work.  But when it comes to
>> something more complicated like compositing into an opengl scene or even as
>> simple as rendering to a sub rect of a window then you're into the realm of
>> vendor extensions and without a port to your specific hardware it probably
>> won't work.  Gstreamer 1.0+ goes some way to addressing these issues, but
>> even within the standard packages there still seems to be multiple APIs for
>> codecs which produce EGL surfaces and none of them have been utilized in
>> QtMultimedia yet.
>>
>>
>> Andrew
>>
>>
>> On Wed, May 25, 2016 at 12:40 AM, Denis Shienkov <
>> denis.shienkov at gmail.com> wrote:
>>
>>> Hi all.
>>>
>>> I have the Toradex (Apalis T30) embedded board:
>>>
>>> https://www.toradex.com/computer-on-modules/apalis-arm-family/nvidia-tegra-3
>>>
>>> which has:
>>>
>>> http://developer.toradex.com/files/toradex-dev/uploads/media/Colibri/Linux/Images/Apalis_T30_LinuxImageV2.6Beta1_20160331.tar.bz2
>>>
>>> Linux Image, with the GStreamer 0.10.
>>>
>>> Also I have compiled Qt 5.6 and installed it to the board.
>>>
>>> When I try to play any *.MP4 video file from the QML Video item:
>>> http://doc.qt.io/qt-5/qml-qtmultimedia-video.html
>>>
>>> I see, that my video has very-very lags, and I see (with htop utility)
>>> that CPU loading is ~300% (
>>> three cores running on ~100%).
>>>
>>> But, when I try to play the video, using the 'nvgstplayer' utility, I
>>> see that CPU loading is ~46% and all fine. Seems, the QML player loads the
>>> SW codecs instead of HW codecs...
>>>
>>> Is there are any way to say to QML player to use a HW codecs? Or, maybe,
>>> I need to fix a sources of QtMM (where I need to change it?) Or, maybe, is
>>> there are another way to do this?
>>>
>>> Otherwise, is just a **HELL** and there is no sense in QML QtMultimedia
>>> at all (But I need QML for anumations). And I do not know, how to fix it
>>> (I'm not expert in GStreamer).
>>>
>>> PS: I have tried the 'gst-launch' pipelines instead of 'nvgstplayer', as
>>> described here:
>>> http://developer.toradex.com/knowledge-base/video-playback-%28linux%29
>>> but it crashes with sigsegv....
>>>
>>> BR,
>>> Denis
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Development mailing list
>>> Development at qt-project.org
>>> http://lists.qt-project.org/mailman/listinfo/development
>>>
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20160525/0857c92a/attachment.html>